"Error in (function (x) : attempt to apply non-function"

[ealonsogzl]

Hi there,

I am receiving the following message using the terra package:
Error in (function (x) : attempt to apply non-function

I call it "message" instead of error because everything seems to work fine. The message appears when I call for the first time in a script a function from the terra package.

I am running the package in a conda installed R version on a HPC cluster. I use to update regularly the package directly from github.

(RH: I moved your second issue to #31)

Cheers

[rhijmans]

There are indeed message that may be ignored, or so it seems. But they are very annoying. It is problem that may occur when using Rcpp modules. It is reported in a few other places, but I have not found a solution.

They are related to garbage collection. Because of this it is difficult to make a reproducible example. The messages do not occur with small toy data sets (as there is no garbage collection, presumably).

Here is an example that I can reproduce or now (from #29)

download.file("https://github.com/geocompr/d/releases/download/1/r1.zip", "r1.zip")
download.file("https://github.com/geocompr/d/releases/download/1/example_route_2d.gpkg", "example_route_2d.gpkg", mode="wb")
unzip("r1.zip")

library(terra)
dem <- terra::rast("r1/hdr.adf")
dem_matrix = terra::xyFromCell(dem, 1:ncell(dem))
v <- vect("example_route_2d.gpkg")
#Error in x$.self$finalize() : attempt to apply non-function
#Error in (function (x)  : attempt to apply non-function
#Error in (function (x)  : attempt to apply non-function

[ealonsogzl]

I will just ignore the messages,
Thanks for the answer!

[rhijmans]

Re-opening because this remains an issue

[rhijmans]

Another example. You need remotes::install_github(cropmodels/Recocrop)

library(Recocrop)
library(terra)
#This is version 0.6.2 of the "terra" package, for evaluation only

logo <- rast(system.file("ex/logo.tif", package="terra"))   
names(logo) <- c("red", "green", "blue")
p <- matrix(c(48, 48, 48, 53, 50, 46, 54, 70, 84, 85, 74, 84, 95, 85, 
    66, 42, 26, 4, 19, 17, 7, 14, 26, 29, 39, 45, 51, 56, 46, 38, 31, 
    22, 34, 60, 70, 73, 63, 46, 43, 28), ncol=2)
a <- matrix(c(22, 33, 64, 85, 92, 94, 59, 27, 30, 64, 60, 33, 31, 9,
    99, 67, 15, 5, 4, 30, 8, 37, 42, 27, 19, 69, 60, 73, 3, 5, 21,
    37, 52, 70, 74, 9, 13, 4, 17, 47), ncol=2)
xy <- rbind(p, a)
e <- extract(logo, xy)
#Error in x$.self$finalize() : attempt to apply non-function
#Error in (function (x)  : attempt to apply non-function
#Error in (function (x)  : attempt to apply non-function

Note that Recocrop imports terra, but Recocrop is not used in any way. If I do not load it, the messages do not occur. Also, if I switch the order from library(Recocrop); library(terra), the error does not occur.

Here is what is actually done under the S4 interface (and some omitted R validity checking)

library(Recocrop)
library(terra)
f <- system.file("ex/logo.tif", package="terra")
ptr <- SpatRaster$new(f)
p <- matrix(c(48, 48, 48, 53, 50, 46, 54, 70, 84, 85, 74, 84, 95, 85, 
   66, 42, 26, 4, 19, 17, 7, 14, 26, 29, 39, 45, 51, 56, 46, 38, 31, 
   22, 34, 60, 70, 73, 63, 46, 43, 28), ncol=2)
a <- matrix(c(22, 33, 64, 85, 92, 94, 59, 27, 30, 64, 60, 33, 31, 9,
   99, 67, 15, 5, 4, 30, 8, 37, 42, 27, 19, 69, 60, 73, 3, 5, 21,
   37, 52, 70, 74, 9, 13, 4, 17, 47), ncol=2)
xy <- rbind(p, a)
i <- ptr$cellFromXY(xy[,1], xy[,2])
e <- ptr$extractCell(i)

But running it like this does not produce the messages.

[rhijmans]

A similar situation is discussed here for the xcms package. Although that package does not use Rcpp, the messages appear to stem from the mzR package, which does use Rcpp (I believe it has to do with Rcpp modules; see discussion on stackoverflow).

[h-a-graham]

Not sure if this helps but I am having this issue and can reproduce it like so...
From a fresh session...

ras_template <- function(){
  
  .emp <- terra::rast(nrows=10000, ncols=10000,
                      xmin=199999.998103917, xmax=299999.999051958,
                      ymin=-0.000744555247365497, ymax=100000.004551688)
  .emp[]<- 0
  return(.emp)
}
ras_template()

This results in the error and seems to never complete. However...

ras_template <- function(){
  
  .emp <- terra::rast(nrows=10000, ncols=10000,
                      xmin=199999.998103917, xmax=299999.999051958,
                      ymin=-0.000744555247365497, ymax=100000.004551688)
  
  return(.emp)
}
.emp <- ras_template()
.emp[]<- 0

This works without errors and is very quick...

Finally though... This does not reproduce the error - could this therefore be an issue relating to the handling of precision?

ras_template <- function(){
  
  .emp <- terra::rast(nrows=10000, ncols=10000,
                      xmin=200000, xmax=300000,
                      ymin=0, ymax=100000)
  .emp[]<- 0
  return(.emp)
}
.emp <- ras_template()

I wish I could be more help in diagnosing!
Thanks for this great package!

[h-a-graham]

Okay.. I just updated to the Dev version of {terra} and this no longer hangs but does occasionally print the error message. The previous version I was running was from CRAN on windows 10.

[aloboa]

I get the error message kind of erratically. Most often I re-run the same command and does not complain
Error in x$.self$finalize() : attempt to apply non-function

Using 1.5.24 on

> Sys.info()
                                                           sysname                                                            release 
                                                           "Linux"                                         "5.15.11-76051511-generic" 
                                                           version                                                           nodename 
"#202112220937~1640185481~21.04~b3a2c21-Ubuntu SMP Mon Jan 3 16:5"                                                           "pop-os" 
                                                           machine                                                              login 
                                                          "x86_64" 

[Kodiologist]

@rhijmans Do you know any way to silence these messages as a workaround? Sometimes I write code that produces a lot of them.

[rhijmans]

You can ignore these messages from the garbage collector. They do not affect your data. They are very annoying. I have done a lot of things to get to the bottom of this, but sofar to no avail. I have much simpler packages that also show these messages and I need to go back to one of these to create a reproducible example for others to look at (even it only happens on the first run) that does not require installation of GDAL etc.

[Kodiologist]

You can ignore these messages from the garbage collector. They do not affect your data.

I know. I was just hoping for a way to cut down on the terminal spam so I can more easily see my own messages.

[tlhenvironment]

You can ignore these messages from the garbage collector. They do not affect your data.

I know. I was just hoping for a way to cut down on the terminal spam so I can more easily see my own messages.

I think you can just wrap the statement with try(..., silent = TRUE), this works for me. Seems like a dirty workaround..
try(terra::XXX, silent = TRUE)

edit: on second thoughts this might not work consistently ~

[phytoclast]

I don't think this error can still be ignored in the latest RStudio (2022.07.1 Build 554). The default behavior is to stop processing scripts upon the first error, even if the error has no consequence in the results.

I've had to run the same script twice to get it to finish, after it stumbled the first time while rasterizing vector spatial data. The raster does get created, but the script stops. But then it seems to work well any subsequent time even after clearing objects from memory.

[rhijmans]

Can you try again with the current development version (1.6-20) and report back?

[phytoclast]

I have installed version 1.6-20 last night. The original script still gets tripped on the same line (though it may have worked the first time). When I remove the offending lines, it trips again on a step subsetting an sf object instead. All the while some spatraster objects are loaded and other functions applies to them. If I move the offending lines to the end, the script didn't trip.

I am not sure it is only a terra issue if it trips on the sf object. There is no reason that the order should matter, because the script works when ran again in the same R session (even with prior object loadings wiped). I should also try it on another computer when I get a chance.

[rhijmans]

Thanks. This is certainly is related to terra, but since this is related to garbage collection the messages may appear at any time. It would be useful if you could share your script and indicate where they occur though (by private email is fine if you do not want to add it here).

[kadyb]

Here is the issue in the RStudio repository to allow the user to choose the option, rather than stopping the script as the default.

As workaround, the previous version of RStudio can be used, or as @tlhenvironment wrote use try(code, silent = TRUE) .

[rhijmans]

Thanks @kadyb but try(xxx, silent=TRUE) does not help here; presumably because these are garbage collector error messages that are perhaps triggered by, but not directly related to the function being called.

[kevinushey]

FWIW I can't reproduce any of the issues reported here on macOS:

> sessionInfo()
R Under development (unstable) (2022-09-18 r82870)
Platform: aarch64-apple-darwin21.6.0 (64-bit)
Running under: macOS Monterey 12.6

Matrix products: default
LAPACK: /opt/homebrew/Cellar/lapack/3.10.1_1/lib/liblapack.3.10.1.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] terra_1.6-20   Recocrop_0.3-2

loaded via a namespace (and not attached):
[1] compiler_4.3.0 tools_4.3.0    meteor_0.3-4   Rcpp_1.0.9   

Is there something I'm missing? Does this issue only reproduce on certain platforms?

[brownag]

@kevinushey

FWIW I can't reproduce any of the issues reported here on macOS:

In my testing (Ubuntu and Windows, using 1.6-19 vs 1.6-20 w/ R 4.2.1) the error-generating code from #218 has been fixed by the added gc() call in 1.6-20.

[rhijmans]

@kevinushey with the current CRAN version, this should still give you the messages.

library("terra")
library("raster")
#terra version 1.2.13
fnz <- "nz_elev.rda"
if (!file.exists(fnz)) {
 download.file("https://github.com/Nowosad/spDataLarge/raw/master/data/nz_elev.rda", fnz, mode = "wb")
}
load("nz_elev.rda")
nz <- rast(nz_elev)
plot(nz)
#Error in (function (x)  : attempt to apply non-function
#Error in x$.self$finalize() : attempt to apply non-function
#Error in x$.self$finalize() : attempt to apply non-function
#Error in x$.self$finalize() : attempt to apply non-function
#Error in (function (x)  : attempt to apply non-function

That is #218 which I fixed yesterday. And with that change (a regular call to gc) I expect that these messages will still pop up now and then, but not nearly as often.

Also, somewhat the example above has been consistently giving these error messages for a long time, whereas other examples have stopped working (or started to work without error messages) for unknown reasons.

[kevinushey]

Thanks; I can reproduce now. I'm trying to see if I can unwind the issue at all...

R reference classes (which Rcpp Modules are) register their finalizers here:

https://github.com/wch/r-source/blob/18d16095f36e28862d125d88659bda28d93d0269/src/library/methods/R/refClass.R#L202-L203

One can poke a little bit into where finalizers are being registered with something like:

trace(reg.finalizer, quote({
  
  classDef <- dynGet("classDef", ifnotfound = NULL)
  if (!is.null(classDef)) {
    browser()
    f <- function(x) {
      writeLines("Running finalizer")
      browser()
    }
  }
  
})

It seems like the finalize method exists when the finalizer is registered...

Browse[3]> e$.self
C++ object <0x7faf4e044f00> of class 'SpatOptions' <0x7faf7f03aee0>
Browse[3]> e$.self$finalize
Class method definition for method finalize()
function () 
{
    .Call(list(name = "CppObject__finalize", address = <pointer: 0x7faf9c798290>, 
        dll = list(name = "Rcpp", path = "/Users/kevin/Library/R/x86_64/4.2/library/Rcpp/libs/Rcpp.so", 
            dynamicLookup = TRUE, handle = <pointer: 0x21d996a80>, 
            info = <pointer: 0x7faf9c7fe910>), numParameters = 2L), 
        <pointer: 0x7faf7f03aee0>, .pointer)
}
<environment: 0x7faf4d70da48>

but that finalize method is gone later when the finalizer method is actually run. Will try to dive in a bit more later.

[kevinushey]

Given that the bug goes away with something as simple as:

trace(reg.finalizer, quote({
  tryCatch(e$.self$finalize, error = identity)
}), print = FALSE)

It's almost surely some kind of GC protection issue. That said, I don't have a good idea as to why the finalize method would be special here; it seems like other methods are just fine, and chasing down protection issues like this is notoriously challenging.

Is there any opportunity for terra to do something similar here? E.g. an initialize() method that finds the finalize() method and bumps its reference count?

[kalibera]

In principle this doesn't have to be a protection issue. It can be also a race condition between the finalizer and something else in the involved packages. When trying to provoke the problem with higher probability, gctorture may help. It may help to check if anything ever is trying to clear the involved pointers (either by reading the code, or by a debugger watchpoint). Perhaps there is code that cleans up the object, some kind of a destructor, including things that the finalizer needs to run.

[kevinushey]

Maybe the issue here is that reference classes initialize methods on instances of their classes lazily, or at least are doing so in this case? For example:

trace(reg.finalizer, quote({
  
  classDef <- dynGet("classDef", ifnotfound = NULL)
  if (!is.null(classDef)) {
    writeLines("# Registering finalizer ----")
    print(is.function(unclass(e$.self)@.xData$finalize))
    print(is.function(e$.self$finalize))
    print(is.function(unclass(e$.self)@.xData$finalize))
  }
  
  f <- function(x) {
    writeLines("# Running finalizer ----")
    x$.self$finalize()
  }
  
}))

library("terra")
library("raster")
#terra version 1.2.13
fnz <- "nz_elev.rda"
if (!file.exists(fnz)) {
 download.file("https://github.com/Nowosad/spDataLarge/raw/master/data/nz_elev.rda", fnz, mode = "wb")
}
load("nz_elev.rda")
nz <- rast(nz_elev)
plot(nz)

With this code, you'll see output like:

Tracing reg.finalizer(selfEnv, function(x) x$.self$finalize(), TRUE) on entry 
# Registering finalizer ----
[1] FALSE
[1] TRUE
[1] TRUE

and no errors will occur when the finalizer is run.

Maybe the attempt to install the finalize instance method when x$.self$finalize is called within a finalizer is too late, or some behavior in the $ method being called is failing to install the class method here?

It seems like methods:::.dollarForEnvRefClass is responsible for inserting the finalize method (via envRefInferField) but that isn't getting called in the finalizer.

[kevinushey]

This also fixes the issue:

trace(reg.finalizer, quote({

  classDef <- dynGet("classDef", ifnotfound = NULL)
  if (!is.null(classDef)) {
    f <- function(x) {
      method <- selectMethod("$", list(x = class(x$.self)))
      finalize <- method(x$.self, "finalize")
      finalize()
    }
  }

}))

In other words, explicitly selecting the appropriate $ method works, but the "regular" $ dispatch seems to fail for some reason. I've reached the limits of my ability to debug S4 dispatch, though...

[kevinushey]

@kalibera, I think this could be a potential patch for R:

===================================================================
--- src/library/methods/R/refClass.R    (revision 82898)
+++ src/library/methods/R/refClass.R    (working copy)
@@ -199,8 +199,10 @@
                 methods::initRefFields(.Object, classDef, selfEnv, list(...))
         }
     }
-    if(is.function(classDef@refMethods$finalize))
+    if(is.function(classDef@refMethods$finalize)) {
+        invisible(selfEnv$.self$finalize)
         reg.finalizer(selfEnv, function(x) x$.self$finalize(), TRUE)
+    }
     lockBinding(".self", selfEnv)
     lockBinding(".refClassDef", selfEnv)
     ## validObject was called from the S4 initialize; check that

This would force the finalize method to be materialized on the .self object when the finalizer is registered. It's sort of a band-aid for whatever the real underlying issue is, though.

[kalibera]

Thanks, that's good to know this makes the bug go away, but we can't add this work-around to base R when we don't know what is the underlying issue (or whether it could be fixed properly, whether is is actually an issue in a package, etc).

[kadyb]

So it looks like this problem has been fixed upstream in {Rcpp} by @kevinushey (thank you very much!). I tested some of my scripts that definitely were returning this error and now it is ok. I used {terra} stable version (1.6-17) and {Rcpp} dev version (1.0.9.3).

As Kevin wrote, more testing and feedback are very welcome. If you want to test it, you need to install:

install.packages("Rcpp", repos = "https://rcppcore.r-universe.dev")

[eddelbuettel]

Thanks -- the install command the Rcpp Core team recommend is

install.packages("Rcpp", repos="https://rcppcore.github.io/drat")

as we make explicit releases to our drat which we consider preferable to random repo snapshots.

[rhijmans]

This indeed appears to be fixed with Rcpp-dev and the current terra-dev (where I have removed the calls to gc). What a relief. I understand that the next CRAN release of Rcpp may be in 2023, so I will restore the band-aid gc calls in the next release of "terra".

[eddelbuettel]

Yes. Getting Rcpp updated at CRAN is a bit of work given the over 2500 reverse depends so we prefer the bi-annual cycle. But we do maintain a curated repository we recommend for use in the interim. Installing from it is a one-liner

install.packages("Rcpp", repos="https://rcppcore.github.io/drat")