Have evaluate_plan handle lists of vectors of indices

[kendonB]

I would like to be able to expand a command by a list of indices. I'm not sure what the best interface is for this but I'd like to be able to do something like:

test <- drake_plan(test = run_it(wc__))
evaluate_plan(test, list(wc__ = list(1:4, 5:8, 9:12)))
#>     target      command
#>1  test_1_4  run_it(1:4)
#>2  test_5_8  run_it(5:8)
#>3 test_9_12 run_it(9:12)

[wlandau-lilly]

That's an excellent point. It's often difficult to know when to break things up into targets and when to combined lots of steps into a single target. You are proposing a happy medium, similar to #77 and #79, but more generalized and with informative names. I want to make this sort of thing easier in drake. Super important for ease of use.

For your use case, what about this?

library(drake)
library(magrittr)
names <- c("1_4", "5_8", "9_12")
values <- gsub("_", ":", names)
test <- drake_plan(test = run_it(wc__)) %>%
  expand_plan(values = names) %>%
  evaluate_plan(wildcard = "wc__", values = values, expand = FALSE)

test
##      target      command
## 1  test_1_4  run_it(1:4)
## 2  test_5_8  run_it(5:8)
## 3 test_9_12 run_it(9:12)

Unfortunately, drake does not have a more automatic way of noticing that you have a list of vectors of consecutive indices and then generating informative names.

[wlandau-lilly]

Better yet:

test <- drake_plan(test = run_it(wc__))
test <- evaluate_plan(test, list(wc__ = drake_strings(1:4, 5:8, 9:12)))
test$target <- gsub(":", "_", test$target)
test

##     target      command
## 1  test_1_4  run_it(1:4)
## 2  test_5_8  run_it(5:8)
## 3 test_9_12 run_it(9:12)

Is that sufficient? We still have that annoying call to gsub() to get rid of the colons, but otherwise, it's more convenient than my original example code.

[kendonB]

My current workaround does something similar to your suggestion, but using paste0 and tibble. The problem was that my solution is an awkward workaround, and yours is still somewhat awkward.

Is it feasible that evaluate_plan could implement my suggestion in a way that's non-breaking?

[kendonB]

Your second solution is less awkward still :)

[kendonB]

Another question: I presume that having colons in target names causes problems, but that may not be true. Do colons in target names cause problems?

[wlandau-lilly]

If you do something like x <- readd("my:target"), you should not have problems using x. But if you loadd(list = "my:target"), then you may have trouble using my:target as a symbol. I thought about using make.names(..., unique = TRUE) in sanitize_plan() to repair all the target names, but it corrupts single-quoted file targets.

make.names("'report.md'")
## [1] "X.report.md."

I am resistant to writing a naive ad-hoc parser of my own.

[kendonB]

I think that a naive ad-hoc parser is the right choice if it outputs a warning. It seems like it would surely avoid more problems than it would cause

…

On Sat, Jan 6, 2018, 10:50 AM Will Landau ***@***.***> wrote: If you do something like x <- readd("my:target"), you should not have problems using x. But if you loadd(list = "my:target"), then you may have trouble using my:target as a symbol. I thought about using make.names(..., unique = TRUE) in sanitize_plan() <https://github.com/wlandau-lilly/drake/blob/bb3734f35fdb7caec709648b12c186a7eca3b63a/R/sanitize.R#L1> to repair all the target names, but it corrupts single-quoted file targets. make.names("'report.md'")## [1] "X.report.md." I am resistant to writing a naive ad-hoc parser of my own. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#187 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFFKkYCnt-HLCv0FURYUf9m-_wYb6_2bks5tHpk3gaJpZM4RU8WF> .

[wlandau-lilly]

In that case, I will include some of the symbols that are likely to come up in this sort of generative templating. I am not going too far just yet because I'm not very good at anticipating weird regexp edge cases.

[wlandau-lilly]

The symbol parser for cleaning up target names is repair_target_names(). And for interpreting lists of numeric vectors, I just coerced values to a character vector in evaluate_plan() and expand_plan(). Your original suggestion now works verbatim, and there is a special unit test.