Skip to content

Commit

Permalink
Make the mirai dispatcher error message extremely verbose
Browse files Browse the repository at this point in the history
  • Loading branch information
wlandau committed Feb 6, 2024
1 parent 3f1d70c commit 9b69ff7
Show file tree
Hide file tree
Showing 3 changed files with 47 additions and 6 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -66,4 +66,4 @@ Language: en-US
VignetteBuilder: knitr
Config/testthat/edition: 3
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.3.0
RoxygenNote: 7.3.1
1 change: 1 addition & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
* Add a new `walk()` method for controllers and controller groups to submit tasks in batch and return control immediately without waiting for any task to complete (#148, @jcheng5).
* Revive the `collect()` method for popping multiple tasks at once (#148, @jcheng5).
* Add controller group methods `nonempty()`, `resolved()`, `unresolved()`, and `unpopped()` to help with #148.
* Make the `mirai` dispatcher error message extremely verbose.

# crew 0.8.0

Expand Down
50 changes: 45 additions & 5 deletions R/utils_mirai.R
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,58 @@ daemons_error <- function(daemons, name) {
try(handle <- ps::ps_handle(pid = pid), silent = TRUE),
"try-error"
)
lines_common <- c(
"Please also try upgrading R packages {nanonext}, {mirai}, and {crew}",
"to their latest versions on CRAN. (Likewise with {targets} if you are",
"using it.) Upgrading these packages solves many kinds of errors.\n\n",
"If that does not help, the {mirai} dispatcher process may have trouble",
"starting, either because it cannot find R or the R package library,",
"or because there is something strange about your file system or",
".Rprofile/.Renviron files, or you have an {renv} project that",
"takes a long time to start. Please make sure R sessions can start",
"quickly in your project. For example, for {renv}, try setting",
"RENV_CONFIG_SANDBOX_ENABLED=false and",
"RENV_CONFIG_SYNCHRONIZED_CHECK=false in your .Renviron file",
"in to speed up {renv} project initialization.\n\n",
"Another possibility is an out-of-memory error.",
"The dispatcher can run out of memory if it is overwhelmed with",
"data objects too large or too many to comfortably fit inside a single",
"R process. As a workaround, each task could save or load large files",
"instead of sending or returning large objects in R.",
"Those large files could either live locally on disk or in a cloud",
"bucket. If you are using {targets}, you might consider",
"storage = \"worker\", retrieval = \"worker\", and/or",
"cloud storage as documented at",
"https://books.ropensci.org/targets/performance.html and",
"https://books.ropensci.org/targets/cloud-storage.html."
)
info <- if_any(
exists,
sprintf(
"dispatcher running at pid %s with status \"%s\". Connection issue?",
paste(
c(
"A {mirai} dispatcher process is running at pid %s with status",
"\"%s\". There may be a network connection issue,",
"or the dispatcher is simply busy managing lots of tasks or",
"uploading/downloading large datasets. Maybe try increasing",
"seconds_timeout in your {crew} controller.\n\n",
lines_common
),
collapse = " "
),
pid,
ps::ps_status(handle)
),
paste(
"The mirai dispatcher is not running. Please call the start() method",
"of the controller (e.g. your_controller$start()",
"before using methods like push(), collect(), and scale().",
"If you already did, then the dispatcher may have exited too early."
c(
"The {mirai} dispatcher is not running. If you are using {crew}",
"without {targets}, be sure to call the start() method of the",
"controller before doing anything else. If you already did,",
"or if you are using {targets}, then the dispatcher process probably",
"started and then failed.\n\n",
lines_common
),
collapse = " "
)
)
crew_error(paste(message, info))
Expand Down

0 comments on commit 9b69ff7

Please sign in to comment.