Add `regex_program` java APIs and unit tests #12548

cindyyuanjiang · 2023-01-14T02:06:02Z

Description

Adds a set of java regex APIs that take in a regex_program as parameter and java unit tests. This is part of the solution for NVIDIA/spark-rapids#7295.

Checklist

I am familiar with the Contributing Guidelines.
New or existing tests cover these changes.
The documentation is up to date with these changes.

Signed-off-by: Cindy Jiang <[email protected]>

rapids-bot · 2023-01-14T02:06:06Z

Pull requests from external contributors require approval from a rapidsai organization member with write or admin permissions before CI can begin.

java/src/main/java/ai/rapids/cudf/CaptureGroups.java

mythrocks · 2023-01-18T20:03:54Z

@cindyyuanjiang: I've added Non-breaking and Feature labels to this PR, so that CI might run.

Signed-off-by: Cindy Jiang <[email protected]>

…g/cudf into regex-program-cudf

mythrocks

A couple of minor nitpicks to start.
I have yet to go over the tests.

java/src/main/java/ai/rapids/cudf/ColumnView.java

java/src/main/java/ai/rapids/cudf/CaptureGroups.java

java/src/main/java/ai/rapids/cudf/RegexProgram.java

mythrocks

A couple more minor nitpicks.

java/src/test/java/ai/rapids/cudf/ColumnVectorTest.java

java/src/main/java/ai/rapids/cudf/RegexFlags.java

java/src/test/java/ai/rapids/cudf/ColumnVectorTest.java

Signed-off-by: Cindy Jiang <[email protected]>

java/src/main/java/ai/rapids/cudf/CaptureGroups.java

java/src/main/java/ai/rapids/cudf/RegexFlags.java

mythrocks

Generally, LGTM. A couple of very minor nitpicks remain.

Signed-off-by: Cindy Jiang <[email protected]>

java/src/main/java/ai/rapids/cudf/ColumnView.java

java/src/main/java/ai/rapids/cudf/RegexFlags.java

java/src/main/java/ai/rapids/cudf/RegexProgram.java

revans2 · 2023-01-19T15:54:03Z

java/src/main/java/ai/rapids/cudf/RegexProgram.java

+  public CaptureGroups capture() {
+    return capture;
+  }
+}


nit: can we add in native APIs to allow us to call the following APIs.

cudf/cpp/include/cudf/strings/regex/regex_program.hpp

Lines 93 to 113 in 22087b3

/**

* @brief Return the number of instructions in this instance

*

* @return Number of instructions

*/

int32_t instructions_count() const;

/**

* @brief Return the number of capture groups in this instance

*

* @return Number of groups

*/

int32_t groups_count() const;

/**

* @brief Return the pattern used to create this instance

*

* @param num_strings Number of strings for computation

* @return Size of the working memory in bytes

*/

std::size_t compute_working_memory_size(int32_t num_strings) const;

I am fine if it is a follow on issue that we do later.

I can definitely add them in a follow up issue. I find the current PR a bit large to track things easily. Thank you for the comment!

Signed-off-by: Cindy Jiang <[email protected]>

…g/cudf into regex-program-cudf

codecov · 2023-01-25T06:57:59Z

Codecov Report

Base: 86.58% // Head: 85.73% // Decreases project coverage by -0.85% ⚠️

Coverage data is based on head (2554907) compared to base (b6dccb3).
Patch has no changes to coverable lines.

Additional details and impacted files

@@               Coverage Diff                @@
##           branch-23.02   #12548      +/-   ##
================================================
- Coverage         86.58%   85.73%   -0.85%     
================================================
  Files               155      155              
  Lines             24368    24889     +521     
================================================
+ Hits              21098    21339     +241     
- Misses             3270     3550     +280

Impacted Files	Coverage Δ
python/cudf/cudf/_version.py	`1.41% <0.00%> (-98.59%)`	⬇️
python/cudf/cudf/core/buffer/spill_manager.py	`72.50% <0.00%> (-7.50%)`	⬇️
python/cudf/cudf/utils/dtypes.py	`77.85% <0.00%> (-1.61%)`	⬇️
python/cudf/cudf/options.py	`86.11% <0.00%> (-1.59%)`	⬇️
python/cudf/cudf/core/single_column_frame.py	`94.30% <0.00%> (-1.27%)`	⬇️
python/cudf/cudf/io/json.py	`91.04% <0.00%> (-1.02%)`	⬇️
...ython/custreamz/custreamz/tests/test_dataframes.py	`98.38% <0.00%> (-1.01%)`	⬇️
python/dask_cudf/dask_cudf/io/csv.py	`96.34% <0.00%> (-1.00%)`	⬇️
python/dask_cudf/dask_cudf/io/parquet.py	`91.81% <0.00%> (-0.59%)`	⬇️
python/cudf/cudf/core/multiindex.py	`91.66% <0.00%> (-0.51%)`	⬇️
... and 47 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

ajschmidt8 · 2023-01-25T14:57:35Z

/ok to test

revans2 · 2023-01-25T16:09:24Z

The test failures appear to be totally unrelated. It failed in a boolean segmented sort test. Not sure if you just need to upmerge or what.

revans2 · 2023-01-25T18:36:48Z

rerun tests

revans2 · 2023-01-25T19:03:27Z

hmm the same tests failed again. I wonder if others are seeing these same failures or not.

revans2 · 2023-01-25T19:08:59Z

@davidwendt it looks like SegmentedSortInt.Bool is failing in this PR only on v100 and this PR didn't touch any code that should be impacting this path. Is the patch just out of date or is there some other issue going on here? Calling you out because I think you added this test after there was a bug in CUB with segmented sort.

ajschmidt8 · 2023-01-25T19:37:59Z

rerun tests

@revans2, the rerun tests command does not work with the new GitHub Actions CI setup.

You can check out the docs below for information on how to rerun tests in GH Actions

https://docs.rapids.ai/resources/github-actions/#rerunning-failed-github-actions

davidwendt · 2023-01-25T19:51:47Z

@davidwendt it looks like SegmentedSortInt.Bool is failing in this PR only on v100 and this PR didn't touch any code that should be impacting this path. Is the patch just out of date or is there some other issue going on here? Calling you out because I think you added this test after there was a bug in CUB with segmented sort.

Yes, this was a cmake error that is fixed by rapidsai/rapids-cmake#353
You will need to do a Rerun All Jobs as referenced by AJ in his comment.

revans2 · 2023-01-26T15:06:54Z

@cindyyuanjiang your branch is still out of date. Please upmerge again. Also because this is deprecating some APIs that the spark plugin uses we are going to need to put up a PR in the plugin that adds the warnings to an allow list so we don't break the build.

You will need to add an annotation similar to

@scala.annotation.nowarn("msg=method XXX in class YYY is deprecated")

You can put this on each off the methods that calls one of the deprecated methods. Sorry I know this is a pain.

ajschmidt8 · 2023-01-26T15:20:22Z

@cindyyuanjiang your branch is still out of date. Please upmerge again

@revans2, a fully up-to-date PR branch is not required for a PR to merge.

If a PR has all of the required reviews and all of the required GitHub checks are passing, then it can be merged by someone with write permissions or greater posting /merge as a comment on the PR.

The Update Branch button is a GitHub feature that we enabled (src) to compliment the new Recently Updated check shown in the screenshot below.

You can read about the Recently Updated check here: https://docs.rapids.ai/resources/recently-updated/

The Update Branch button and the Recently Updated check are separate but complimentary features.

At this time, the Recently Updated check is not required, but that may change in the future.

revans2 · 2023-01-26T15:23:42Z

@ajschmidt8 you are right. I just got confused when I saw

But it is not a blocker....

revans2 · 2023-01-26T15:24:18Z

/merge

cindyyuanjiang · 2023-01-26T18:11:26Z

@cindyyuanjiang your branch is still out of date. Please upmerge again. Also because this is deprecating some APIs that the spark plugin uses we are going to need to put up a PR in the plugin that adds the warnings to an allow list so we don't break the build.

@revans2 Thank you very much! I will follow up with the spark plugin side.

revans2 · 2023-01-26T18:46:21Z

@cindyyuanjiang it's OK I have a patch for the warnings almost ready to go. I realized I hit merge a bit too soon on this so I figured I would take the hit and make the patch

#12548 introduced a number of GPU resource leaks in ColumnVectorTest. This cleans them up by wrapping them in `try` blocks. Authors: - Jason Lowe (https://github.com/jlowe) Approvers: - Robert (Bobby) Evans (https://github.com/revans2) - Jim Brennan (https://github.com/jbrennan333) - Nghia Truong (https://github.com/ttnghia) URL: #12625

This reverts commit 20c945b.

* Revert "Fix leaks in ColumnVectorTest (#12625)" This reverts commit fb17ac7. * Revert "Add `regex_program` java APIs and unit tests (#12548)" This reverts commit 20c945b. --------- Co-authored-by: Cindy Jiang <[email protected]> Co-authored-by: GALI PREM SAGAR <[email protected]>

added regex_program APIs and relevant java classes

6d3947d

Signed-off-by: Cindy Jiang <[email protected]>

github-actions bot added the Java Affects Java cuDF API. label Jan 14, 2023

cindyyuanjiang and others added 4 commits January 13, 2023 18:10

Merge branch 'branch-23.02' into regex-program-cudf

037f7bf

Merge branch 'branch-23.02' into regex-program-cudf

a164846

Merge branch 'branch-23.02' into regex-program-cudf

e04f9ea

Merge branch 'branch-23.02' into regex-program-cudf

722622b

mythrocks self-requested a review January 18, 2023 19:27

mythrocks reviewed Jan 18, 2023

View reviewed changes

java/src/main/java/ai/rapids/cudf/CaptureGroups.java Outdated Show resolved Hide resolved

mythrocks reviewed Jan 18, 2023

View reviewed changes

java/src/main/java/ai/rapids/cudf/CaptureGroups.java Outdated Show resolved Hide resolved

mythrocks added feature request New feature or request non-breaking Non-breaking change labels Jan 18, 2023

cindyyuanjiang added 2 commits January 18, 2023 12:27

fixed code formatting

9c5b19b

Signed-off-by: Cindy Jiang <[email protected]>

Merge branch 'regex-program-cudf' of https://github.com/cindyyuanjian…

a0e3a8b

…g/cudf into regex-program-cudf

mythrocks requested changes Jan 18, 2023

View reviewed changes

java/src/main/java/ai/rapids/cudf/ColumnView.java Show resolved Hide resolved

java/src/main/java/ai/rapids/cudf/CaptureGroups.java Outdated Show resolved Hide resolved

java/src/main/java/ai/rapids/cudf/RegexProgram.java Outdated Show resolved Hide resolved

mythrocks requested changes Jan 18, 2023

View reviewed changes

cindyyuanjiang and others added 2 commits January 18, 2023 15:22

removed getter function from java classes and fixed code formatting

a4d3a42

Signed-off-by: Cindy Jiang <[email protected]>

Merge branch 'branch-23.02' into regex-program-cudf

c36d9ab

mythrocks reviewed Jan 19, 2023

View reviewed changes

java/src/main/java/ai/rapids/cudf/CaptureGroups.java Outdated Show resolved Hide resolved

mythrocks reviewed Jan 19, 2023

View reviewed changes

java/src/main/java/ai/rapids/cudf/RegexFlags.java Outdated Show resolved Hide resolved

mythrocks approved these changes Jan 19, 2023

View reviewed changes

changed RegexFlags and CaptureGroups to private constructors

5348acf

Signed-off-by: Cindy Jiang <[email protected]>

cindyyuanjiang marked this pull request as ready for review January 19, 2023 01:14

cindyyuanjiang requested a review from a team as a code owner January 19, 2023 01:14

cindyyuanjiang changed the title ~~[WIP] Add regex_program java APIs and unit tests~~ Add regex_program java APIs and unit tests Jan 19, 2023

revans2 reviewed Jan 19, 2023

View reviewed changes

cindyyuanjiang added 3 commits January 19, 2023 16:27

updated stringSplit and stringSplitRecord APIs

dc38a80

Signed-off-by: Cindy Jiang <[email protected]>

updated containsRe and matchesRe apis

fa3e52d

Signed-off-by: Cindy Jiang <[email protected]>

updated extractAllRecord API and java unit tests

4cff615

Signed-off-by: Cindy Jiang <[email protected]>

cindyyuanjiang added 2 commits January 24, 2023 10:57

removed empty pattern check for stringSplit APIs

1f9ae3c

Signed-off-by: Cindy Jiang <[email protected]>

Merge branch 'regex-program-cudf' of https://github.com/cindyyuanjian…

6c8a53e

…g/cudf into regex-program-cudf

revans2 approved these changes Jan 25, 2023

View reviewed changes

Merge branch 'branch-23.02' into regex-program-cudf

2554907

rapids-bot bot merged commit 20c945b into rapidsai:branch-23.02 Jan 26, 2023

cindyyuanjiang mentioned this pull request Jan 26, 2023

Add warnings to an allowlist for deprecated Java regex APIs NVIDIA/spark-rapids#7594

Closed

revans2 mentioned this pull request Jan 26, 2023

Fix deprecation warnings NVIDIA/spark-rapids#7596

Merged

cindyyuanjiang deleted the regex-program-cudf branch January 26, 2023 19:00

jlowe mentioned this pull request Jan 26, 2023

Fix leaks in ColumnVectorTest #12625

Merged

3 tasks

cindyyuanjiang added a commit to cindyyuanjiang/cudf that referenced this pull request Jan 28, 2023

Revert "Add regex_program java APIs and unit tests (rapidsai#12548)"

09e640e

This reverts commit 20c945b.

cindyyuanjiang added a commit to cindyyuanjiang/cudf that referenced this pull request Jan 28, 2023

Revert "Add regex_program java APIs and unit tests (rapidsai#12548)"

608963f

This reverts commit 20c945b.

cindyyuanjiang mentioned this pull request Jan 28, 2023

Revert regex program java APIs and tests #12639

Merged

3 tasks

cindyyuanjiang mentioned this pull request Feb 13, 2023

[FEA] Switch to regex_program APIs NVIDIA/spark-rapids#7730

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `regex_program` java APIs and unit tests #12548

Add `regex_program` java APIs and unit tests #12548

cindyyuanjiang commented Jan 14, 2023 •

edited

Loading

rapids-bot bot commented Jan 14, 2023

mythrocks commented Jan 18, 2023

mythrocks left a comment

mythrocks left a comment

mythrocks left a comment

revans2 Jan 19, 2023

cindyyuanjiang Jan 21, 2023

codecov bot commented Jan 25, 2023 •

edited

Loading

ajschmidt8 commented Jan 25, 2023

revans2 commented Jan 25, 2023

revans2 commented Jan 25, 2023

revans2 commented Jan 25, 2023

revans2 commented Jan 25, 2023 •

edited

Loading

ajschmidt8 commented Jan 25, 2023

davidwendt commented Jan 25, 2023

revans2 commented Jan 26, 2023

ajschmidt8 commented Jan 26, 2023

revans2 commented Jan 26, 2023

revans2 commented Jan 26, 2023

cindyyuanjiang commented Jan 26, 2023 •

edited

Loading

revans2 commented Jan 26, 2023

	/**
	* @brief Return the number of instructions in this instance
	*
	* @return Number of instructions
	*/
	int32_t instructions_count() const;

	/**
	* @brief Return the number of capture groups in this instance
	*
	* @return Number of groups
	*/
	int32_t groups_count() const;

	/**
	* @brief Return the pattern used to create this instance
	*
	* @param num_strings Number of strings for computation
	* @return Size of the working memory in bytes
	*/
	std::size_t compute_working_memory_size(int32_t num_strings) const;

Add regex_program java APIs and unit tests #12548

Add regex_program java APIs and unit tests #12548

Conversation

cindyyuanjiang commented Jan 14, 2023 • edited Loading

Description

Checklist

rapids-bot bot commented Jan 14, 2023

mythrocks commented Jan 18, 2023

mythrocks left a comment

Choose a reason for hiding this comment

mythrocks left a comment

Choose a reason for hiding this comment

mythrocks left a comment

Choose a reason for hiding this comment

revans2 Jan 19, 2023

Choose a reason for hiding this comment

cindyyuanjiang Jan 21, 2023

Choose a reason for hiding this comment

codecov bot commented Jan 25, 2023 • edited Loading

Codecov Report

ajschmidt8 commented Jan 25, 2023

revans2 commented Jan 25, 2023

revans2 commented Jan 25, 2023

revans2 commented Jan 25, 2023

revans2 commented Jan 25, 2023 • edited Loading

ajschmidt8 commented Jan 25, 2023

davidwendt commented Jan 25, 2023

revans2 commented Jan 26, 2023

ajschmidt8 commented Jan 26, 2023

revans2 commented Jan 26, 2023

revans2 commented Jan 26, 2023

cindyyuanjiang commented Jan 26, 2023 • edited Loading

revans2 commented Jan 26, 2023

Add `regex_program` java APIs and unit tests #12548

Add `regex_program` java APIs and unit tests #12548

cindyyuanjiang commented Jan 14, 2023 •

edited

Loading

codecov bot commented Jan 25, 2023 •

edited

Loading

revans2 commented Jan 25, 2023 •

edited

Loading

cindyyuanjiang commented Jan 26, 2023 •

edited

Loading