Recursive gc error

[hadley]

library(dplyr)
library(nycflights13)
library(magrittr)

path <- tempfile()
dir.create(path)

gctorture(TRUE)

flights %>% 
  group_by(month) %>% 
  group_walk(~ vroom::vroom_write(.x, sprintf("%s/month-%02i.csv", path, .y$month)))

*** recursive gc invocation
*** recursive gc invocation
Error in vroom_write_(xx, file, delim, eol, na_str = na, col_names = col_names,  :
  cannot get ALTSTRING_ELT during GC
Error: no more error handlers available (recursive errors?); invoking 'abort' restart
sive gc invo>c aion
** ecrsvegcinoctin
** ecursive gc invocation
*** recursive gc invocation
*** recursive gc invocation
*** recursive gc invocation
*** recursive gc invocation
*** recursive gc invocation
*** recursive gc invocation
Error: no more error handlers available (recursive errors?); invoking 'abort' restart
Error: no more error handlers available (recursive errors?); invoking 'abort' restart
Error: no more error handlers available (recursive errors?); invoking 'abort' restart
Error: no more error handlers available (recursive errors?); invoking 'abort' restart
Error: no more error handlers available (recursive errors?); invoking 'abort' restart

From tidyverse/multidplyr#127 (comment)

[DavisVaughan]

Possibly related is #270, which claims that all ALTREP columns are converted to non-ALTREP before writing

[DavisVaughan]

I think the problem is with the POSIXct column.

vroom calls vroom_convert() to materialize any ALTREP columns here, but then immediately calls output_column() to perform any necessary type conversion before writing. The POSIXct->character conversion will actually create another ALTREP wrapper object.

https://github.com/r-lib/vroom/blob/0ea9d8c96eade582a695632a3be5dc45c7e0fff4/R/vroom_write.R#L66-L69

# Make it large enough to trigger an ALTREP wrapper
x <- rep(as.POSIXct("2019-01-01", "UTC"), 1000)

# This makes an ALTREP character vector, oops
.Internal(inspect(vroom:::output_column(x)))
#> @7fefac305790 16 STRSXP g0c0 []  wrapper [srt=-2147483648,no_na=0]
#>   @7fef6b10a000 16 STRSXP g0c7 [REF(1)] (len=1000, tl=0)
#>     @7fefabfc4c98 09 CHARSXP g0c3 [REF(1001),gp=0x60] [ASCII] [cached] "2019-01-01T00:00:00Z"
#>     @7fefabfc4c98 09 CHARSXP g0c3 [REF(1001),gp=0x60] [ASCII] [cached] "2019-01-01T00:00:00Z"
#>     @7fefabfc4c98 09 CHARSXP g0c3 [REF(1001),gp=0x60] [ASCII] [cached] "2019-01-01T00:00:00Z"
#>     @7fefabfc4c98 09 CHARSXP g0c3 [REF(1001),gp=0x60] [ASCII] [cached] "2019-01-01T00:00:00Z"
#>     @7fefabfc4c98 09 CHARSXP g0c3 [REF(1001),gp=0x60] [ASCII] [cached] "2019-01-01T00:00:00Z"
#>     ...

It looks to me like it would be safe to just swap the ordering of these two calls, and that should fix it.

I'm trying to run the original example to try and reproduce the issue right now. Very slow!

[DavisVaughan]

Slightly more minimal reprex:

library(nycflights13)
library(vroom)

path <- tempfile(fileext = ".csv")

gctorture(TRUE)

vroom_write(flights, path)

---

And I confirmed that this does not fail, so I think my analysis is correct

library(nycflights13)
library(vroom)

path <- tempfile(fileext = ".csv")

flights$time_hour <- NULL

gctorture(TRUE)

vroom_write(flights, path)