lwgeom not on latest build

[Robinlovelace]

After pulling-down the latest version of the image (see below), I get:

> library(sf)
Linking to GEOS 3.5.1, GDAL 2.1.2, PROJ 4.9.3
> library(lwgeom)
Error in library(lwgeom) : there is no package called ‘lwgeom’

May be fixed with #16

Docker image pull message:

PS E:\repos> docker pull rocker/geospatial
Using default tag: latest
latest: Pulling from rocker/geospatial
54f7e8ac135a: Already exists
021dd68904f6: Pull complete
8d3e964cf18d: Pull complete
81bbbedbef33: Pull complete
c7878d347b3f: Pull complete
a7aae3dbd958: Pull complete
da950bb11a69: Pull complete
9802011fbe26: Pull complete
334f65727953: Pull complete
e506f9195a59: Pull complete
Digest: sha256:c3158846628b07d4616fafa30ef1d97cdc1b706d1c9aa30d62e7862ffd75703e
Status: Downloaded newer image for rocker/geospatial:latest
PS E:\repos> docker run -e PASSWORD=notrealpassword -d -p 8788:8787 -v ${pwd}:/home/rstudio/data rocker/geospatial
fb494143ce66c7cf45d9e4415c5fbad699d5626a950e17abe3eac6d23afdd39d

[Robinlovelace]

Error message on install.packages("lwgeom"):

lwgeom.cpp: In function ‘Rcpp::LogicalVector CPL_is_polygon_cw(Rcpp::List)’:
lwgeom.cpp:232:46: error: ‘lwgeom_is_clockwise’ was not declared in this scope
     out[i] = lwgeom_is_clockwise(lwgeom_cw[i]);
                                              ^
/usr/local/lib/R/etc/Makeconf:171: recipe for target 'lwgeom.o' failed
make: *** [lwgeom.o] Error 1
ERROR: compilation failed for package ‘lwgeom’

[Robinlovelace]

Update: the problem is solved with

devtools::install_github("r-spatial/lwgeom")

In case this is useful info, heads-up @edzer + would be useful to know when the latest version is likely to be on CRAN. I think that will fix this issue and that #16 may not be needed (but maybe worth explicitly installing lwgeom, an important package).

[edzer]

I guess I'll have to wait until mid Feb for the 2-monthly minimum release cycle - it is not a bug of lwgeom, is it?

[Robinlovelace]

Not sure. All I know is that the CRAN version fails to install but the dev version succeeds. Progress!

[cboettig]

@Robinlovelace @edzer Thanks as always for your watchful attention to these things!

Yup, looks like the CRAN version of lwgeom cannot install on the rocker image but the GitHub version is okay. Looks like this may be due to differences in the configure file for the CRAN version and GitHub version -- which one could argue is a bug in lwgeom. I'd certainly welcome another release of lwgeom as I think that would resolve this issue; I think CRAN would accept that argument but who knows.

Note that the Rocker image uses only the debian:stable (debian:9) versions of the system libraries such as libgeos-dev and liblwgeom-dev, which are by now probably behind what you test with on Travis using the ubuntugis/ubuntugis-unstable PPA. I'm deliberately avoiding adding PPAs or apt-pinning to unstable in the rocker-versioned stack, including geospatial, to have as stable and reproducible an environment as possible (note that prior to R 3.4, the versioned stack builds on debin:jessie and builds these libraries from tagged source versions -- we could do that here if newer versions are necessary; I don't want to depend on an unstable upstream binary where I can no longer ensure the build uses the same version).

[edzer]

liblwgeom is much less of a shared resource than PROJ, GEOS and GDAL, so it's not a surprise that an older liblwgeom on this distro doesn't allow compiling against it. It looks like the PostGIS people are pulling back from distributing liblwgeom, and that's why the upcoming lwgeom will ship with its own liblwgeom copy and only use that.

devtools::install_github("r-spatial/lwgeom")

works fine.

[cboettig]

Hi @edzer . Thanks, yes, we've both confirmed install_github works fine above, it's just the current CRAN version that is broken for a stable Debian distribution. Good to know that the GitHub version is shipping it's own copy now! Would you suggest we drop the liblwgeom-dev dependency entirely or is it used by other packages?

We could switch the Dockerfiles here to use:

devtools::install_github("r-spatial/lwgeom@1cd8e1bac5c3ae8ce56ef5368b8a87efa00e716d") 

to maintain a stable / reproducible build in the Dockerfiles, but I really would prefer to rely on released CRAN versions / MRAN snapshots rather than start pinning to arbitrary GitHub hashes.

p.s. it would be pretty straight forward to set Travis up to run additional tests with a stable debian/ubuntu environment (e.g. by running rocker or some other container), usually easier than the elaborate travis config already required. Is that something you would be interested in? I think the ubuntu PPA is great and all, but as someone who cares about reproducibility, I'm not sure there's any simple and consistent way to recreate the version of the software libraries that may have been at ubuntu-unstable at any particular time.

[edzer]

Yes, liblwgeom-dev can be dropped. AFAICT, the ubuntugis-unstable folks only package released libraries, it's hard to miss what that is. The only thing they occasionally mess up is packaging-specific stuff like what geos-config --clibs gives as output, which then breaks our configure scripts.

[cboettig]

@edzer Thanks! Once the next lwgeom hits CRAN we'll do that then.

Yes, I was not suggesting that the PPA was giving pre-release or unstable versions, only that it is not clear to me that it is version-stable. A week or a year from now the the PPA will not be giving the exact same versions of the software that it is giving today, whereas a year from now rocker/geospatial:3.5.1 will continue to use the precise same version of the libraries that it uses today, and thus provides users a consistent and reproducible environment.

Anyway, it's great this bug is already addressed in the current dev version of lwgeom and hope it hits CRAN soon! thanks again for all the great work.

[edzer]

Just for clarification: the binary image of rocker/geospatial:3.5.1 has this property, but does the image built from a Dockerfile at some point in the future have it too? Do you pin GEOS/GDAL/PROJ versions in debian distributions?

[cboettig]

This is just a property of most linux releases. The debian:stretch (aka debian:9) release will always have the same version of software, same version of compilers, etc in its default repositories. These get patches and 'security updates', but never update versions. (Of course newer versions of things can be either built from source or installed from additional repositories). The rocker-versioned stack builds on specific debian releases for this reason. The rocker r-base stack takes the opposite strategy, building on debian:testing (with some debian:unstable mixed in), as those names imply, those are sliding repos that aren't version-stable. Software from the debian:unstable repos has already been released by the original maintainers, (it's not pre-release stuff), it just hasn't finished the checks by the debian maintainers who will then move it to testing or the next stable release, (every 6 mo for ubuntu, or every ~ 2 years for debian!) e.g. R 3.5.2 is in debian:unstable, whereas debian:stable is still on 3.4.4 I think. guessing R will be at 3.5.3 or maybe 3.6.0 by the time of the next debian:stable release, debian:10, probably this summer). Dirk would know these details much better than me, but this is already probably more than you wanted! hth

[edzer]

I understand now, thanks a lot!

[colman-humphrey]

Hey @cboettig (and @edzer) - I'm sure you're both way ahead of me on this one, but I had no trouble building the latest CRAN version of lwgeom (0.1-7, from 2019-05-06) on top of the current rocker/geospatial image, or on top of a devel image built with GEOS 3.7.0.

I'm guessing this isn't big news since above it's clear the latest lwgeom as of Jan worked already and was just waiting for release to CRAN, which has happened.

[Robinlovelace]

Thanks for the info @colman-humphrey. I therefore think this issue can be closed.

[colman-humphrey]

Hey @Robinlovelace that makes sense - although given that it all works, lwgeom can be brought back into the image. All good either way with me!

[Robinlovelace]

Good point @colman-humphrey - I've opened a PR: #26