[R-package] lightgbm removed from CRAN

[jameslamb]

Description

As of this morning, {lightgbm} has been removed from CRAN.

https://cran.r-project.org/web/packages/lightgbm/index.html

Package ‘lightgbm’ was removed from the CRAN repository.

Formerly available versions can be obtained from the archive.

Archived on 2021-10-25 as check problems were not corrected in time.

A summary of the most recent check results can be obtained from the check results archive.

Please use the canonical form https://CRAN.R-project.org/package=lightgbm to link to this page.

Additional Comments

"check problems" most likely refers to #4674 and #4680.

[jameslamb]

This is going to be VERY disruptive for users of the R package 😬

@shiyu1994 did you receive any emails from CRAN in the last few weeks asking us to re-submit? If so can you please post them here in their entirety (with dates and times) so we can understand why the package has been removed?

I can open a v3.3.1 release pull request today, but the last time the package was taken down by CRAN, we were told we could not re-submit for a month 😬

[llrs]

Hi! Sorry to hear that the package was removed from CRAN. I'm in no way to speak for CRAN but in case it helps I've heard that you can re-submit your package with more frequency than once a month if the changes are to fix problems found on CRAN checks. Hope it can be made available soon!

[shiyu1994]

After searching my email, I found this was sent on Oct 12th,

It seems that today is the deadline for fixing the problem. I'm sorry about missing that email. I'll be careful next time.

[???? [email protected] ?????????? http://aka.ms/LearnAboutSenderIdentification?????????????]

Dear maintainer,

Please see the problems shown on
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fweb%2Fchecks%2Fcheck_results_lightgbm.html&data=04%7C01%7Cyushi2%40microsoft.com%7Ce45fee760332456a232608d98cd54d41%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637695670177785045%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=R%2Bt3eXZ7gA92SstlqOTv5yBIgQ8JL%2FluW8n1qLt7Yh4%3D&reserved=0.

Please correct before 2021-10-25 to safely retain your package on CRAN.

Do remember to look at the 'Additional issues'.

The CRAN Team

[shiyu1994]

I should have noticed the email and remind the team to fix the problems before today. Sorry about making such a mistake.

[jameslamb]

Thanks for the quick response @shiyu1994 . It's ok, these things happen!

I'll open up a PR for a new release today.

[mayer79]

@jameslamb : I was quick enough to test the 3.3 version on CRAN and it worked very well, fantastic work! No hard crashs as with the old version. I fear those CRAN e-mails as they sometimes tend to even end up in the spam folder... As @llrs wrote, I don't think there is a problem to immediately resubmit a fixed version.

This is the "additional issues" link that is mentioned by Brian:
https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-ASAN/lightgbm/

[jameslamb]

I don't think there is a problem to immediately resubmit a fixed version

Thanks, for sure! I wanted to wait to see what the exact email from CRAN said, since in the past they have explicitly told us not to resubmit for a fixed amount of time. Since @shiyu1994 hasn't received any such emails saying there is a limit on when we can resubmit, I hope we'll be able to submit today if @shiyu1994 and @StrikerRUS have time to review #4715 and the PRs linked there.

This is the "additional issues" link that is mentioned by Brian:

Yep, thanks! You might be interested to see...I have an open PR to replicate CRAN's gcc-ASAN, gcc-UBSAN, clang-ASAN, and clang-UBSAN tests in this project's continuous integration: #4678

Thanks to the amazing work by @wch and collaborators on https://github.com/wch/r-debug.

[AdrianAntico]

I just noticed this issue as well. I tried to install from github and received the following error message, in case this helps in any way...

> devtools::install_github('microsoft/LightGBM/R-package', upgrade = FALSE, dependencies = FALSE, force = TRUE)
Downloading GitHub repo microsoft/LightGBM@HEAD
"C:\PROGRA~1\Git\cmd\git.exe" clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/boostorg/compute C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14/microsoft-LightGBM-adef94f/R-package/../external_libs/compute
"C:\PROGRA~1\Git\cmd\git.exe" clone --depth 1 --no-hardlinks --recurse-submodules https://gitlab.com/libeigen/eigen.git C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14/microsoft-LightGBM-adef94f/R-package/../external_libs/eigen
"C:\PROGRA~1\Git\cmd\git.exe" clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/fmtlib/fmt.git C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14/microsoft-LightGBM-adef94f/R-package/../external_libs/fmt
"C:\PROGRA~1\Git\cmd\git.exe" clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/lemire/fast_double_parser.git C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14/microsoft-LightGBM-adef94f/R-package/../external_libs/fast_double_parser
√  checking for file 'C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14\microsoft-LightGBM-adef94f\R-package/DESCRIPTION' ...
-  preparing 'lightgbm': (560ms)
E  checking DESCRIPTION meta-information ... 
   Malformed package version.
   
   See section 'The DESCRIPTION file' in the 'Writing R Extensions'
   manual.
Error: Failed to install 'lightgbm' from GitHub:
  System command 'Rcmd.exe' failed, exit status: 1, stdout + stderr:
E> * checking for file 'C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14\microsoft-LightGBM-adef94f\R-package/DESCRIPTION' ... OK
E> * preparing 'lightgbm':
E> * checking DESCRIPTION meta-information ... ERROR
E> Malformed package version.
E> 
E> See section 'The DESCRIPTION file' in the 'Writing R Extensions'
E> manual.

[jameslamb]

@AdrianAntico you cannot use devtools::install_github() / remotes::install_github() to install {lightgbm}.

The file structure in source control here is not a valid, self-contained R package. For example, the core C/C++ code in the project is stored in src/ at the top-level of the repo, and then copied at build time into the relevant directories when building the R and Python packages.

See https://github.com/microsoft/LightGBM/tree/master/R-package#contents for a list of available options for installing the R package.

We will try to restore the R package to CRAN as soon as possible.

[AdrianAntico]

Ah, makes sense

[leecommamichael]

This hurts a lot. Why is there a 1 month wait for resubmission? That makes zero sense.

[llrs]

@leecommamichael For fixes there isn't a 1 month wait for resubmitting. That period is just if there are updates that do not fix the package on issues detected by CRAN checks. Check this message from one of the CRAN volunteer:

Then policies you are quoting have two reasons:

1. we take a look what is going on if a submission comes in too quickly
   as this is typically suspicious. Hence a quick submission creates manual
   work in any case,

2. each version of a package is archived, so when people submit
   frequently, CRAN is growing quickly.

As CRAN is a repository for releases software, we expect that releases
are well tested and hence do not happen too frequently.

In case you nevertheless found a serious bug, please submit a fixed
version and explain why this was needed in the submission comments.

[leecommamichael]

@leecommamichael For fixes there isn't a 1 month wait for resubmitting. That period is just if there are updates that do not fix the package on issues detected by CRAN checks. Check this message from one of the CRAN volunteer:

Then policies you are quoting have two reasons:

1. we take a look what is going on if a submission comes in too quickly
   as this is typically suspicious. Hence a quick submission creates manual
   work in any case,
2. each version of a package is archived, so when people submit
   frequently, CRAN is growing quickly.

As CRAN is a repository for releases software, we expect that releases
are well tested and hence do not happen too frequently.
In case you nevertheless found a serious bug, please submit a fixed
version and explain why this was needed in the submission comments.

I appreciate the detail, Lluís. I assumed from earlier discussion, and the fact that it has been many days without the package in CRAN, that we'd be waiting a full month.

[shiyu1994]

We've uploaded R package for the new release 3.3.1 to CRAN. And we are waiting for a manual check.

[jameslamb]

Anyone wanting to track the progress of {lightgbm}'s latest submission to CRAN is encouraged to use https://lockedata.github.io/cransays/articles/dashboard.html, maintained by @maelle and others.

At the moment it looks like {lightgbm} has been placed in the same manual-review queue as completely new packages.

There are 77 total incoming submissions to CRAN being evaluated right now. I don't know how to turn that into an estimate of how long it will take for the package to be restored.

Those who really really cannot wait can install directly from the most recent artifact published to GitHub (but this will mean a build from source, as we don't prepare pre-compiled binaries like CRAN does).

install.packages("https://github.com/microsoft/LightGBM/releases/download/v3.3.1/lightgbm-3.3.1-r-cran.tar.gz")

I got that URL from https://github.com/microsoft/LightGBM/releases/tag/v3.3.1.

[llrs]

Yes, once packages are archived they are considered as new and must go under the manual inspection again. I made some analysis of time to approval of new packages here, expect 1 week but it is highly variable so it could be just 1 or 2 days.

[mikemahoney218]

Those who really really cannot wait can install directly from the most recent artifact published to GitHub (but this will mean a build from source, as we don't prepare pre-compiled binaries like CRAN does).

install.packages("https://github.com/microsoft/LightGBM/releases/download/v3.3.1/lightgbm-3.3.1-r-cran.tar.gz")

If the goal is to install any version of lightgbm (and not specifically the new 3.3.1 version), a second option would be to install from the MRAN snapshot from the day before archival:

install.packages(
  "lightgbm",
  repos = "https://cran.microsoft.com/snapshot/2014-10-24/"
  )

I have no idea how critical the sanitizer checks actually are (so if using 3.3.0 would be actively discouraged), but this installation method takes advantage of the binary packages.

[Javdat]

Those who really really cannot wait can install directly from the most recent artifact published to GitHub (but this will mean a build from source, as we don't prepare pre-compiled binaries like CRAN does).

install.packages("https://github.com/microsoft/LightGBM/releases/download/v3.3.1/lightgbm-3.3.1-r-cran.tar.gz")

I got that URL from https://github.com/microsoft/LightGBM/releases/tag/v3.3.1.

Note: you might need to specify repos=NULL otherwise R might search for package with that URL name and fail:

install.packages(
  "https://github.com/microsoft/LightGBM/releases/download/v3.3.1/lightgbm-3.3.1-r-cran.tar.gz",
  repos = NULL
)

[StrikerRUS]

At the moment it looks like {lightgbm} has been placed in the same manual-review queue as completely new packages.

Our status has been changed to pending which means

pending: the CRAN maintainers are waiting for an action on your side. You should check your emails!

@shiyu1994 Please check your e-mail.

[shiyu1994]

The last email I got from CRAN is this, which is sent 3 hours after I confirmed the submission.

It seems there's no action being asked to take? Following is the log file and the link the the check results.

[00details.log[24792].log]
(https://github.com/microsoft/LightGBM/files/7438441/_00details.log_.24792.log)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2Fincoming_pretest%2Flightgbm_3.3.1_20211028_035536%2F&data=04%7C01%7Cyushi2%40microsoft.com%7C501cb39749f94b20d82b08d999cb3880%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637709920840925512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=12OpqFovY8ovbw22DcWv4jlydHaERb8p5KMC7DFhUzk%3D&reserved=0

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2Fincoming_pretest%2Flightgbm_3.3.1_20211028_035536%2FWindows%2F00check.log&data=04%7C01%7Cyushi2%40microsoft.com%7C501cb39749f94b20d82b08d999cb3880%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637709920840925512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=Jtu9GNWqkC8aeUqBKJ%2BMfo%2FQ%2F2Lg7zI%2BhDKn041zIIs%3D&reserved=0

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2Fincoming_pretest%2Flightgbm_3.3.1_20211028_035536%2FDebian%2F00check.log&data=04%7C01%7Cyushi2%40microsoft.com%7C501cb39749f94b20d82b08d999cb3880%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637709920840925512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=quuN4L48V3XeIMCtrk2bXTsQJAR3Avo7VvVVuZbtRGc%3D&reserved=0

[jameslamb]

thanks @shiyu1994 ! That's a good sign...the two jobs referenced in those links had no issues.

[StrikerRUS]

Looks like v3.3.1 release is starting to roll out at the CRAN 🤞

https://cran.r-project.org/web/checks/check_results_lightgbm.html

[shiyu1994]

Great to hear that!

[jameslamb]

A few more checks (linux, mac, solaris) have passed, and clang-ASAN and gcc-ASAN issues have been removed from the package's checks page.

https://cran.r-project.org/web/checks/check_results_lightgbm.html

I don't know if it's possible to see any information for the "Additional Checks" (like gcc-ASAN) that says "CRAN ran this and it passed"...in my experience, you only get to learn that those checks ran if they fail.

At the moment, according to https://cran.r-project.org/web/packages/lightgbm/index.html, there are v3.3.1 binaries available for armg64 mac environments, and otherwise the package can be installed from source.

Still a lot of checks to go, but at least now it is once again possible to get {lightgbm} from CRAN with install.packages("lightgbm", repos = "https://cran.r-project.org")!

[jameslamb]

According to https://cran.r-project.org/web/packages/lightgbm/index.html, binaries for Mac are now available.

No checks have been run yet on Windows, I think related to some expected maintenance on CRAN's Windows cluster today. From r-pkg-devel mailing list (https://stat.ethz.ch/pipermail/r-package-devel/2021q4/007498.html):

Dear all,

winbuilder service and CRAN incoming checks (submission is possible)
will be offline tomorrow (Nov 1) from 10 am CET to not later than 6pm.

Some CRAN hardware in Dortmund will be physically migrated to a new
location.

Best,
Uwe Ligges
(CRAN team)

[jameslamb]

Checks going well so far. Windows binaries for R 4.2.x 4.1.x are now available.

Comparing https://cran.r-project.org/web/checks/check_results_lightgbm.html to https://cran.r-project.org/web/checks/check_results_xgboost.html, I think the following checks remain:

• r-devel-linux-x86_64-debian-clang
• r-devel-windows-x86_64
• r-devel-windows-x86_64-gcc10-UCRT
• r-patched-linux-x86_64
• r-release-linux-x86_64
• r-oldrel-windows-ix86+x86_64

As well as any of CRAN's "Additional Checks" (link), whose status we can't see.

[mayer79]

Excellent, thank you so much!

[jameslamb]

According to https://cran.r-project.org/web/checks/check_results_lightgbm.html, {lightgbm} 3.3.1 is now passing all of CRAN's checks, and CRAN has finished preparing Windows and Mac binaries.

Thanks to everyone who helped with this issue, and apologies for the disruption!

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed.
To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues
including a reference to this.