[FEA] Add predict_proba() to XGBoost-style models in FIL C++ #2894

levsnv · 2020-10-01T06:44:02Z

No description provided.

GPUtester · 2020-10-01T06:44:25Z

Please update the changelog in order to start CI tests.

View the gpuCI docs here.

cpp/src/fil/fil.cu

cpp/src/fil/infer.cu

cpp/test/sg/fil_test.cu

cpp/src/fil/infer.cu

levsnv · 2020-10-08T22:00:37Z

To copy the Slack response: I definitely agree that there can be a neat single template for softmax, and I will implement the suggestions above

cpp/src/fil/infer.cu

levsnv · 2020-10-15T06:51:58Z

cpp/src/fil/infer.cu

-using BlockReduceHost =
-  typename cub::BlockReduce<vec<NITEMS, float>, FIL_TPB,
-                            cub::BLOCK_REDUCE_WARP_REDUCTIONS, 1, 1, 600>;
+size_t block_reduce_footprint_host() {


I was changing half the cases where this was used, as well as deleting the device-side template using statements. This meant the host ones were left alone and did not need to define class separately from footprint. This allowed to implement smem footprint additions for GROVE_PER_CLASS_* in a much more readable way (see below).

cpp/src/fil/fil.cu

cpp/src/fil/infer.cu

levsnv · 2020-11-03T08:23:42Z

now conflicts with #3088 for combo ML::fil::output_t combo values (but a simple update to resolve merge conflict)

…ition

This reverts commit 8581222.

levsnv · 2021-03-04T04:46:40Z

depends on #3582

levsnv · 2021-03-05T05:15:31Z

python/cuml/test/test_fil.py

-        # FIL doesn't yet support predict_proba() for multi-class
-        # TODO: Add a test for predict_proba() when it's supported
-        gbm_preds = bst.predict(X)
-        gbm_preds = gbm_preds.argmax(axis=1)


since lightgbm doesn't support probabilities without the sklearn API, using it here for both predictions

levsnv · 2021-03-05T22:08:13Z

rerun tests
./test/ml: symbol lookup error: ./test/ml: undefined symbol: _ZN5faiss3gpu20StandardGpuResources20setCudaMallocWarningEb

dantegd · 2021-03-09T16:30:38Z

@JohnZed PR looks good to me, ready to merge and has addressed all feedback, when you have a sec could you take a look?

JohnZed · 2021-03-09T22:05:32Z

@gpucibot merge

codecov-io · 2021-03-10T01:17:18Z

Codecov Report

Merging #2894 (a2eb0a7) into branch-0.19 (fd9ec89) will increase coverage by 0.03%.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           branch-0.19    #2894      +/-   ##
===============================================
+ Coverage        80.70%   80.74%   +0.03%     
===============================================
  Files              227      227              
  Lines            17615    17737     +122     
===============================================
+ Hits             14217    14322     +105     
- Misses            3398     3415      +17

Flag	Coverage Δ
dask	`45.30% <ø> (+0.31%)`	⬆️
non-dask	`72.89% <ø> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
python/cuml/cluster/kmeans.pyx	`91.95% <ø> (ø)`
python/cuml/linear_model/linear_regression.pyx	`88.23% <0.00%> (-3.53%)`	⬇️
...ython/cuml/thirdparty_adapters/sparsefuncs_fast.py	`49.70% <0.00%> (-1.81%)`	⬇️
python/cuml/common/import_utils.py	`58.49% <0.00%> (-0.56%)`	⬇️
...cuml/_thirdparty/sklearn/utils/skl_dependencies.py	`53.39% <0.00%> (-0.53%)`	⬇️
...on/cuml/_thirdparty/sklearn/preprocessing/_data.py	`62.88% <0.00%> (-0.23%)`	⬇️
...ython/cuml/_thirdparty/sklearn/utils/validation.py	`22.37% <0.00%> (-0.08%)`	⬇️
python/cuml/dask/solvers/cd.py	`100.00% <0.00%> (ø)`
python/cuml/common/numba_utils.py	`0.00% <0.00%> (ø)`
python/cuml/internals/global_settings.py	`100.00% <0.00%> (ø)`
... and 35 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0967a00...49fc29d. Read the comment docs.

levsnv force-pushed the softmax branch from 63a2f44 to c8e79c1 Compare October 5, 2020 22:17

levsnv changed the title ~~[WIP] add predict_proba() to XGBoost-style models in FIL C++~~ [WIP] add predict_proba() to XGBoost-style models in FIL C++ [skip-ci] Oct 5, 2020

levsnv changed the title ~~[WIP] add predict_proba() to XGBoost-style models in FIL C++ [skip-ci]~~ [WIP] add predict_proba() to XGBoost-style models in FIL C++ Oct 8, 2020

canonizer suggested changes Oct 8, 2020

View reviewed changes

levsnv commented Oct 14, 2020

View reviewed changes

cpp/src/fil/infer.cu Outdated Show resolved Hide resolved

added softmax to FIL C++

20cb06b

levsnv changed the base branch from branch-0.16 to branch-0.17 October 15, 2020 06:04

levsnv force-pushed the softmax branch from 0c2d7cf to 20cb06b Compare October 15, 2020 06:05

changelog

90f45e6

levsnv requested a review from canonizer October 15, 2020 06:46

levsnv added the 4 - Waiting on Reviewer Waiting for reviewer to review or respond label Oct 15, 2020

levsnv commented Oct 15, 2020

View reviewed changes

canonizer suggested changes Oct 15, 2020

View reviewed changes

levsnv added 4 - Waiting on Author Waiting for author to respond to review and removed 4 - Waiting on Reviewer Waiting for reviewer to review or respond labels Oct 16, 2020

levsnv changed the title ~~[WIP] add predict_proba() to XGBoost-style models in FIL C++~~ [REVIEW] add predict_proba() to XGBoost-style models in FIL C++ Oct 16, 2020

levsnv added 3 commits October 20, 2020 16:08

moved global_bias adjustment from SOTFMAX to be thread-safe

0e73b5e

improved [Vv]ectorized templates, added /=

bdac044

fix bugs

b8b2653

levsnv added 8 commits November 13, 2020 20:09

Merge remote-tracking branch 'rapidsai/branch-0.17' into softmax

e7fba50

updated enum output_t

5c69fe1

started using block strided iterator

6a1d2c9

refactored softmax

e707f29

tried with thrust::iterator_adaptor

4d8c7f8

created TransformedArrayView, moved stride definition to offset defin…

f8aae1c

…ition

tried all_map_reduce_shmem, not much use

8581222

Revert "tried all_map_reduce_shmem, not much use"

69fa665

This reverts commit 8581222.

levsnv added 4 - Waiting on Author Waiting for author to respond to review and removed 4 - Waiting on Reviewer Waiting for reviewer to review or respond labels Mar 2, 2021

levsnv added 3 commits March 1, 2021 20:34

Merge remote-tracking branch 'rapidsai/branch-0.19' into softmax

f9dc5ce

switching to absolute tolerances

03f7f8c

copyright year

0c429c5

levsnv added 2 commits March 4, 2021 19:43

unify thresholds as per @canonizer's suggestion

4e61c31

Merge branch 'absolute-proba-discrepancies' into softmax

3515ce1

levsnv commented Mar 5, 2021

View reviewed changes

levsnv added 3 - Ready for Review Ready for review by team and removed 4 - Waiting on Author Waiting for author to respond to review labels Mar 5, 2021

levsnv mentioned this pull request Mar 5, 2021

[REVIEW] Test FIL probabilities with absolute error thresholds in python #3582

Merged

levsnv assigned JohnZed and levsnv Mar 9, 2021

levsnv added 4 - Waiting on Reviewer Waiting for reviewer to review or respond and removed 3 - Ready for Review Ready for review by team labels Mar 9, 2021

Merge branch 'branch-0.19' of github.com:rapidsai/cuml into softmax

a2eb0a7

JohnZed approved these changes Mar 9, 2021

View reviewed changes

levsnv added 4 - Waiting on Author Waiting for author to respond to review and removed 4 - Waiting on Reviewer Waiting for reviewer to review or respond 4 - Waiting on Author Waiting for author to respond to review labels Mar 9, 2021

test softmax function against xgboost as well

49fc29d

rapids-bot bot merged commit 8b78fa3 into rapidsai:branch-0.19 Mar 10, 2021

levsnv deleted the softmax branch March 10, 2021 04:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEA] Add predict_proba() to XGBoost-style models in FIL C++ #2894

[FEA] Add predict_proba() to XGBoost-style models in FIL C++ #2894

levsnv commented Oct 1, 2020

GPUtester commented Oct 1, 2020

levsnv commented Oct 8, 2020

levsnv Oct 15, 2020

levsnv commented Nov 3, 2020 •

edited

Loading

levsnv commented Mar 4, 2021

levsnv Mar 5, 2021

levsnv commented Mar 5, 2021

dantegd commented Mar 9, 2021

JohnZed commented Mar 9, 2021

codecov-io commented Mar 10, 2021

[FEA] Add predict_proba() to XGBoost-style models in FIL C++ #2894

[FEA] Add predict_proba() to XGBoost-style models in FIL C++ #2894

Conversation

levsnv commented Oct 1, 2020

GPUtester commented Oct 1, 2020

levsnv commented Oct 8, 2020

levsnv Oct 15, 2020

Choose a reason for hiding this comment

levsnv commented Nov 3, 2020 • edited Loading

levsnv commented Mar 4, 2021

levsnv Mar 5, 2021

Choose a reason for hiding this comment

levsnv commented Mar 5, 2021

dantegd commented Mar 9, 2021

JohnZed commented Mar 9, 2021

codecov-io commented Mar 10, 2021

Codecov Report

levsnv commented Nov 3, 2020 •

edited

Loading