R unit tests are not run in CI

[jameslamb]

Are the R package's tests run anywhere in our CI? I looked through Travis, Azure, and Appveyor and as far as I can tell they aren't.

I ask because I just submitted #2334 which is passing all checks, but running the unit tests on the R package fails locally for me.

After noticing that, I saw that even on master the R unit tests locally for me.

Steps to reproduce

Clone lightgbm repo, then from the root of the repo:

# build the package
Rscript build.R

# run the tests
cd R-package/tests
Rscript testthat.R

I get one test failure when doing this:

── 1. Error: training continuation works (@test_basic.R#71)  ───────────────────
(list) object cannot be coerced to type 'double'
1: lgb.get.eval.result(bst2, "train", "binary_logloss", 10) at testthat/test_basic.R:71

Environment Information

Mac OS

R session

R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS  10.14.5

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets 
[6] methods   base     

other attached packages:
[1] lightgbm_2.2.4 R6_2.3.0.9000  testthat_2.2.1

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.0.1      rex_1.1.2         rstudioapi_0.7   
 [4] magrittr_1.5      covr_3.2.1        usethis_1.5.1    
 [7] devtools_2.1.0    pkgload_1.0.2     rlang_0.3.1      
[10] tools_3.5.0       pkgbuild_1.0.4    data.table_1.12.0
[13] sessioninfo_1.1.1 cli_1.1.0         withr_2.1.2.9000 
[16] remotes_2.1.0     lazyeval_0.2.1    assertthat_0.2.1 
[19] digest_0.6.18     rprojroot_1.3-2   crayon_1.3.4     
[22] processx_3.4.1    callr_3.3.1       fs_1.3.1         
[25] ps_1.3.0          glue_1.3.0        memoise_1.1.0    
[28] compiler_3.5.0    desc_1.2.0        backports_1.1.2  
[31] prettyunits_1.0.2 jsonlite_1.6 

[StrikerRUS]

Are the R package's tests run anywhere in our CI? I looked through Travis, Azure, and Appveyor and as far as I can tell they aren't.

You are absolutely right! They aren't.

Are you planning to work on this feature request in the near future? If not, then, according to a new feature requests moderation policy, we should add this to #2302 and close this issue.

[jameslamb]

I am planning to work on it this week. But either way, could you help me understand why this would get added to #2302 ? It's not a "feature" as I understand that, since it's not a user-facing change. Is #2302 supposed to hold "everything except bugs and questions"?

[StrikerRUS]

Is #2302 supposed to hold "everything except bugs and questions"?

I don't know exactly, but according to that, for example, More tests, Add pkgdown documentation support and Rewrite R demos, are there, I think yes.
Ping @guolinke for the help in clarifying #2302 contents.

[guolinke]

yes, 2302 is for everything except bug and question.

[jameslamb]

Ok thanks, will close and add it to that issue. I plan to pick it up soon!

[jameslamb]

Reopening since we have an active PR for this (#2936)

[jameslamb]

just noting here this is now officially, fully closed by #2965