All models failed to converge! with vignette data #141

xec-cm · 2022-08-31T15:02:33Z

Since a few days ago, code that runs corncob that used to work with the same data now returns an error. I think it could be some modification on some corncob dependency.

I have reproduced the error with an example of the corncob vignette:

library(corncob)
data(soil_phylum_small)

soil <- soil_phylum_small

set.seed(1)
da_analysis <- differentialTest(
  formula = ~ DayAmdmt,
  phi.formula = ~ DayAmdmt,
  formula_null = ~ 1,
  phi.formula_null = ~ DayAmdmt,
  test = "Wald",
  boot = FALSE,
  data = soil,
  fdr_cutoff = 0.05
)
#> Error in differentialTest(formula = ~DayAmdmt, phi.formula = ~DayAmdmt, : All models failed to converge! 
#> 
#>            If you are seeing this, it is likely that your model is overspecified. This occurs when your sample size is not large enough to estimate all the parameters of your model. This is most commonly due to categorical variables that include many categories. 
#> 
#>            Alternatively, double-check your values for the arguments `link`, `phi.link`, and `method` to makes sure that they follow the specified options. 
#> 
#>            To confirm you have fixed the issue, try running a model for a single taxon with bbdml.

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.1.2 (2021-11-01)
#>  os       macOS Big Sur 10.16
#>  system   x86_64, darwin17.0
#>  ui       X11
#>  language (EN)
#>  collate  es_ES.UTF-8
#>  ctype    es_ES.UTF-8
#>  tz       Europe/Madrid
#>  date     2022-08-31
#>  pandoc   2.18 @ /Applications/RStudio.app/Contents/MacOS/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package            * version      date (UTC) lib source
#>  ade4                 1.7-19       2022-04-19 [1] CRAN (R 4.1.2)
#>  ape                  5.6-2        2022-03-02 [1] CRAN (R 4.1.2)
#>  assertthat           0.2.1        2019-03-21 [1] CRAN (R 4.1.0)
#>  Biobase              2.54.0       2021-10-26 [1] Bioconductor
#>  BiocGenerics         0.40.0       2021-10-26 [1] Bioconductor
#>  biomformat           1.22.0       2021-10-26 [1] Bioconductor
#>  Biostrings           2.62.0       2021-10-26 [1] Bioconductor
#>  bitops               1.0-7        2021-04-24 [1] CRAN (R 4.1.0)
#>  cachem               1.0.6        2021-08-19 [1] CRAN (R 4.1.0)
#>  callr                3.7.2        2022-08-22 [1] CRAN (R 4.1.2)
#>  cli                  3.3.0        2022-04-25 [1] CRAN (R 4.1.2)
#>  cluster              2.1.4        2022-08-22 [1] CRAN (R 4.1.2)
#>  codetools            0.2-18       2020-11-04 [1] CRAN (R 4.1.2)
#>  colorspace           2.0-3        2022-02-21 [1] CRAN (R 4.1.2)
#>  corncob            * 0.2.0        2021-03-11 [1] CRAN (R 4.1.0)
#>  crayon               1.5.1        2022-03-26 [1] CRAN (R 4.1.2)
#>  data.table           1.14.2       2021-09-27 [1] CRAN (R 4.1.0)
#>  DBI                  1.1.3        2022-06-18 [1] CRAN (R 4.1.2)
#>  detectseparation     0.3          2022-08-26 [1] CRAN (R 4.1.2)
#>  devtools             2.4.4        2022-07-20 [1] CRAN (R 4.1.2)
#>  digest               0.6.29       2021-12-01 [1] CRAN (R 4.1.0)
#>  dplyr                1.0.9        2022-04-28 [1] CRAN (R 4.1.2)
#>  ellipsis             0.3.2        2021-04-29 [1] CRAN (R 4.1.0)
#>  evaluate             0.16         2022-08-09 [1] CRAN (R 4.1.2)
#>  fansi                1.0.3        2022-03-24 [1] CRAN (R 4.1.2)
#>  fastmap              1.1.0        2021-01-25 [1] CRAN (R 4.1.0)
#>  foreach              1.5.2        2022-02-02 [1] CRAN (R 4.1.2)
#>  fs                   1.5.2        2021-12-08 [1] CRAN (R 4.1.0)
#>  generics             0.1.3        2022-07-05 [1] CRAN (R 4.1.2)
#>  GenomeInfoDb         1.30.1       2022-01-30 [1] Bioconductor
#>  GenomeInfoDbData     1.2.7        2021-11-15 [1] Bioconductor
#>  ggplot2              3.3.6        2022-05-03 [1] CRAN (R 4.1.2)
#>  glue                 1.6.2        2022-02-24 [1] CRAN (R 4.1.2)
#>  gtable               0.3.0        2019-03-25 [1] CRAN (R 4.1.0)
#>  highr                0.9          2021-04-16 [1] CRAN (R 4.1.0)
#>  htmltools            0.5.3        2022-07-18 [1] CRAN (R 4.1.2)
#>  htmlwidgets          1.5.4        2021-09-08 [1] CRAN (R 4.1.0)
#>  httpuv               1.6.5        2022-01-05 [1] CRAN (R 4.1.2)
#>  igraph               1.3.4        2022-07-19 [1] CRAN (R 4.1.2)
#>  IRanges              2.28.0       2021-10-26 [1] Bioconductor
#>  iterators            1.0.14       2022-02-05 [1] CRAN (R 4.1.2)
#>  jsonlite             1.8.0        2022-02-22 [1] CRAN (R 4.1.2)
#>  knitr                1.40         2022-08-24 [1] CRAN (R 4.1.2)
#>  later                1.3.0        2021-08-18 [1] CRAN (R 4.1.0)
#>  lattice              0.20-45      2021-09-22 [1] CRAN (R 4.1.2)
#>  lifecycle            1.0.1        2021-09-24 [1] CRAN (R 4.1.0)
#>  lpSolveAPI           5.5.2.0-17.8 2022-08-12 [1] CRAN (R 4.1.2)
#>  magrittr             2.0.3        2022-03-30 [1] CRAN (R 4.1.2)
#>  MASS                 7.3-58.1     2022-08-03 [1] CRAN (R 4.1.2)
#>  Matrix               1.4-1        2022-03-23 [1] CRAN (R 4.1.2)
#>  memoise              2.0.1        2021-11-26 [1] CRAN (R 4.1.0)
#>  mgcv                 1.8-40       2022-03-29 [1] CRAN (R 4.1.2)
#>  mime                 0.12         2021-09-28 [1] CRAN (R 4.1.0)
#>  miniUI               0.1.1.1      2018-05-18 [1] CRAN (R 4.1.0)
#>  multtest             2.50.0       2021-10-26 [1] Bioconductor
#>  munsell              0.5.0        2018-06-12 [1] CRAN (R 4.1.0)
#>  nlme                 3.1-159      2022-08-09 [1] CRAN (R 4.1.2)
#>  numDeriv             2016.8-1.1   2019-06-06 [1] CRAN (R 4.1.0)
#>  permute              0.9-7        2022-01-27 [1] CRAN (R 4.1.2)
#>  phyloseq             1.38.0       2021-10-26 [1] Bioconductor
#>  pillar               1.8.1        2022-08-19 [1] CRAN (R 4.1.2)
#>  pkgbuild             1.3.1        2021-12-20 [1] CRAN (R 4.1.0)
#>  pkgconfig            2.0.3        2019-09-22 [1] CRAN (R 4.1.0)
#>  pkgload              1.3.0        2022-06-27 [1] CRAN (R 4.1.2)
#>  plyr                 1.8.7        2022-03-24 [1] CRAN (R 4.1.2)
#>  prettyunits          1.1.1        2020-01-24 [1] CRAN (R 4.1.0)
#>  processx             3.7.0        2022-07-07 [1] CRAN (R 4.1.2)
#>  profvis              0.3.7        2020-11-02 [1] CRAN (R 4.1.0)
#>  promises             1.2.0.1      2021-02-11 [1] CRAN (R 4.1.0)
#>  ps                   1.7.1        2022-06-18 [1] CRAN (R 4.1.2)
#>  purrr                0.3.4        2020-04-17 [1] CRAN (R 4.1.0)
#>  R.cache              0.16.0       2022-07-21 [1] CRAN (R 4.1.2)
#>  R.methodsS3          1.8.2        2022-06-13 [1] CRAN (R 4.1.2)
#>  R.oo                 1.25.0       2022-06-12 [1] CRAN (R 4.1.2)
#>  R.utils              2.12.0       2022-06-28 [1] CRAN (R 4.1.2)
#>  R6                   2.5.1        2021-08-19 [1] CRAN (R 4.1.0)
#>  Rcpp                 1.0.9        2022-07-08 [1] CRAN (R 4.1.2)
#>  RCurl                1.98-1.8     2022-07-30 [1] CRAN (R 4.1.2)
#>  registry             0.5-1        2019-03-05 [1] CRAN (R 4.1.0)
#>  remotes              2.4.2        2021-11-30 [1] CRAN (R 4.1.0)
#>  reprex               2.0.2        2022-08-17 [1] CRAN (R 4.1.2)
#>  reshape2             1.4.4        2020-04-09 [1] CRAN (R 4.1.0)
#>  rhdf5                2.38.1       2022-03-10 [1] Bioconductor
#>  rhdf5filters         1.6.0        2021-10-26 [1] Bioconductor
#>  Rhdf5lib             1.16.0       2021-10-26 [1] Bioconductor
#>  rlang                1.0.4        2022-07-12 [1] CRAN (R 4.1.2)
#>  rmarkdown            2.16         2022-08-24 [1] CRAN (R 4.1.2)
#>  ROI                  1.0-0        2020-08-31 [1] CRAN (R 4.1.0)
#>  ROI.plugin.lpsolve   1.0-1        2021-06-15 [1] CRAN (R 4.1.0)
#>  rstudioapi           0.14         2022-08-22 [1] CRAN (R 4.1.2)
#>  S4Vectors            0.32.4       2022-03-29 [1] Bioconductor
#>  scales               1.2.1        2022-08-20 [1] CRAN (R 4.1.2)
#>  sessioninfo          1.2.2        2021-12-06 [1] CRAN (R 4.1.0)
#>  shiny                1.7.2        2022-07-19 [1] CRAN (R 4.1.2)
#>  slam                 0.1-50       2022-01-08 [1] CRAN (R 4.1.2)
#>  stringi              1.7.8        2022-07-11 [1] CRAN (R 4.1.2)
#>  stringr              1.4.1        2022-08-20 [1] CRAN (R 4.1.2)
#>  styler               1.7.0        2022-03-13 [1] CRAN (R 4.1.2)
#>  survival             3.4-0        2022-08-09 [1] CRAN (R 4.1.2)
#>  tibble               3.1.8        2022-07-22 [1] CRAN (R 4.1.2)
#>  tidyselect           1.1.2        2022-02-21 [1] CRAN (R 4.1.2)
#>  urlchecker           1.0.1        2021-11-30 [1] CRAN (R 4.1.0)
#>  usethis              2.1.6        2022-05-25 [1] CRAN (R 4.1.2)
#>  utf8                 1.2.2        2021-07-24 [1] CRAN (R 4.1.0)
#>  vctrs                0.4.1        2022-04-13 [1] CRAN (R 4.1.2)
#>  vegan                2.6-2        2022-04-17 [1] CRAN (R 4.1.2)
#>  withr                2.5.0        2022-03-03 [1] CRAN (R 4.1.2)
#>  xfun                 0.32         2022-08-10 [1] CRAN (R 4.1.2)
#>  xtable               1.8-4        2019-04-21 [1] CRAN (R 4.1.0)
#>  XVector              0.34.0       2021-10-26 [1] Bioconductor
#>  yaml                 2.3.5        2022-02-21 [1] CRAN (R 4.1.2)
#>  zlibbioc             1.40.0       2021-10-26 [1] Bioconductor
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

^{Created on 2022-08-31 with reprex v2.0.2}

The text was updated successfully, but these errors were encountered:

statdivlab/corncob#141

xec-cm · 2022-09-01T07:46:38Z

The corncob dependence detectseparation has a new version for 5 days in cran (v0.3). In this new version the way to obtain the separation value is different from v0.2. Therefore these lines of code from the bbdm function fail:

https://github.com/bryandmartin/corncob/blob/7f146f2afcec703d3f21958ef55f48c4fd50a751/R/bbdml.R#L121
https://github.com/bryandmartin/corncob/blob/7f146f2afcec703d3f21958ef55f48c4fd50a751/R/bbdml.R#L129
https://github.com/bryandmartin/corncob/blob/7f146f2afcec703d3f21958ef55f48c4fd50a751/R/bbdml.R#L141
https://github.com/bryandmartin/corncob/blob/7f146f2afcec703d3f21958ef55f48c4fd50a751/R/bbdml.R#L149

In my opinion there are two solutions. The first is to indicate the version of detectseparation as v0.2 in the DESCRIPTION. The other is to adapt the code to the new version of the detectseparation package.

I hope you can resolve this issue early.

Thanks

statdivlab/corncob#141

jorondo1 · 2022-09-01T18:18:15Z

Hi,
I am having a similar problem with the vignette data:

> corncob1_da <- bbdml(formula = OTU.1 ~ DayAmdmt,
+                     phi.formula = ~ DayAmdmt,
+                     data = soil)
Error in if (detectseparation::detect_separation(y = cbind(W, M - W),  : 
  argument is of length zero

I am using the demo script that was presented at STAMPS
https://raw.githubusercontent.com/statdivlab/stamps2022/main/Sunday-afternoon/labs/corncob_tutorial/corncob_tutorial.R

You seem to have fixed something yesterday, I made sure I had the most recent version of corncob but that doesn't seem to work either.

xec-cm · 2022-09-01T19:50:22Z

@jorondo1 I am not the maintainer of this repository... My workaround (as long as Corncob is not updated) was to force installion of the version 0.2 of the detectseparation package. You can do this easily with devtools or remotes:

## Using remotes
remotes::install_version("detectseparation", version = 0.2)

## Using devtools
devtools::install_version("detectseparation", version = 0.2)

After installation, you just need to restart R and run the code normally.

adw96 · 2022-09-01T21:20:29Z

Thanks for posting this issue and your workaround @xec-cm, and for corroborating it, @jorondo1 ! Please note that the software authors have taken note of this issue and are working to come up with a long term solution soon. We will post updates here.

cc @bryandmartin

jorondo1 · 2022-09-06T12:28:31Z

Thank you both, the workaround fixed it for me as well. Cheers !

statdivlab/corncob#141 (comment)

* Updates to workflow for multiple datasets * Fix #27 * Update GH Action * Move workflow structure * updated gh actions * update installation * Update Makefile * update songbird env * add wflow dest to qadabra cli * Change correlation to kendall * add stratification to repeated kfold * update README * new test data * Pin r-detectseparation & r-base statdivlab/corncob#141 (comment)

bryandmartin · 2022-09-21T18:42:04Z

Fixed in 365add6

xec-cm added a commit to MicrobialGenomics-IrsicaixaOrg/dar that referenced this issue Aug 31, 2022

by pass bug with corncob

84491e8

statdivlab/corncob#141

xec-cm added a commit to MicrobialGenomics-IrsicaixaOrg/dar that referenced this issue Sep 1, 2022

force detectseparation v0.2 (corncob bug)

aa339ad

statdivlab/corncob#141

xec-cm added a commit to MicrobialGenomics-IrsicaixaOrg/dar that referenced this issue Sep 1, 2022

force detectseparation v0.2 (corncob bug)

d0ac8f6

statdivlab/corncob#141

xec-cm added a commit to MicrobialGenomics-IrsicaixaOrg/dar that referenced this issue Sep 1, 2022

force detectseparation v0.2 (corncob bug)

63f22af

statdivlab/corncob#141

adw96 added the urgent this needs to happen ASAP label Sep 1, 2022

gibsramen added a commit to biocore/qadabra that referenced this issue Sep 12, 2022

Pin r-detectseparation

8ee738c

statdivlab/corncob#141 (comment)

david-barnett mentioned this issue Sep 20, 2022

CRAN problem fixes #142

Merged

bryandmartin closed this as completed Sep 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

All models failed to converge! with vignette data #141

All models failed to converge! with vignette data #141

xec-cm commented Aug 31, 2022

xec-cm commented Sep 1, 2022

jorondo1 commented Sep 1, 2022 •

edited

Loading

xec-cm commented Sep 1, 2022

adw96 commented Sep 1, 2022

jorondo1 commented Sep 6, 2022

bryandmartin commented Sep 21, 2022

All models failed to converge! with vignette data #141

All models failed to converge! with vignette data #141

Comments

xec-cm commented Aug 31, 2022

xec-cm commented Sep 1, 2022

jorondo1 commented Sep 1, 2022 • edited Loading

xec-cm commented Sep 1, 2022

adw96 commented Sep 1, 2022

jorondo1 commented Sep 6, 2022

bryandmartin commented Sep 21, 2022

jorondo1 commented Sep 1, 2022 •

edited

Loading