address jo-schie's comments (#57)

- Update paper for JOSS submission (fix #53, fix #54, fix #62). - Update vignette with new section on local scale analyses (fix #53). - Update `wdpa_fetch()` function to use the webdriver package for obtaining data (replacing Rselenium package as a dependency) (fix #63). - Update `st_repair_geometry()` to be more robust. - Fix failing tests for `st_repair_geometry()` function. - Update documentation for `wdpa_clean()` function. - Fix broken URL in vignette. - Fix CI.
prioritizr · Sep 28, 2022 · d075533 · d075533
1 parent 40e8e79
commit d075533
Show file tree

Hide file tree

Showing 60 changed files with 903 additions and 232 deletions.
diff --git a/.github/workflows/R-CMD-check-macos.yaml b/.github/workflows/R-CMD-check-macos.yaml
@@ -56,7 +56,7 @@ jobs:
           cache-version: 4
           extra-packages: |
             any::rcmdcheck
-            any::XML
+            cran/XML
           needs: |
             check
 

diff --git a/.github/workflows/R-CMD-check-ubuntu.yaml b/.github/workflows/R-CMD-check-ubuntu.yaml
@@ -46,21 +46,32 @@ jobs:
           r-version: ${{ matrix.config.r }}
           use-public-rspm: true
 
+      - name: Install system dependencies
+        run: |
+          sudo add-apt-repository ppa:ubuntugis/ubuntugis-unstable
+          sudo apt update
+          sudo apt-get install \
+            libgdal-dev libgeos-dev libproj-dev \
+            libgmp3-dev libmpfr-dev libudunits2-dev
+
       - uses: r-lib/actions/setup-r-dependencies@v2
         with:
-          cache-version: 4
+          cache-version: 5
           extra-packages: |
-            any::covr
             any::rcmdcheck
-            any::XML
+            any::covr
+            any::remotes
             prepr=?ignore
+            cran/XML
           needs: |
             check
-            coverage
 
       - name: Install spatial R packages
         run: |
-          pak::pkg_install("dickoa/prepr")
+          options(repos = "https://cloud.r-project.org/")
+          install.packages("sf")
+          install.packages("lwgeom")
+          remotes::install_github("dickoa/prepr")
         shell: Rscript {0}
 
       - name: Session information

diff --git a/.github/workflows/documentation.yaml b/.github/workflows/documentation.yaml
@@ -57,15 +57,19 @@ jobs:
           cache-version: 4
           extra-packages: |
             any::rcmdcheck
-            any::XML
             any::urlchecker
+            any::remotes
             prepr=?ignore
+            cran/XML
           needs: |
             check
 
       - name: Install spatial R packages
         run: |
-          pak::pkg_install("dickoa/prepr")
+          options(repos = "https://cloud.r-project.org/")
+          install.packages("sf")
+          install.packages("lwgeom")
+          remotes::install_github("dickoa/prepr")
         shell: Rscript {0}
 
       - name: Session information

diff --git a/.gitignore b/.gitignore
@@ -9,7 +9,6 @@ doc
 *.Rproj.user
 *.Rhistory
 .Rproj.user
-inst/scripts/global-example-script.Rout
 
 # log files
 gurobi.log
@@ -19,19 +18,28 @@ spell.log
 gp.log
 examples.log
 global-example-script.Rout
-paper/paper.docx
+inst/scripts/global-example-script.Rout
+
 
 # system files
 .directory
+
 # do not ignore the following files
 !.gitkeep
 !*/
 !inst/doc/*.html
 !inst/doc/*.Rmd
 !inst/doc/.gitkeep
+
 # devtools files
 doc/*
 Meta/*
 .build.timestamp
 /doc/
 /Meta/
+
+# paper files
+paper/paper.docx
+/paper/paper_cache/
+/paper/paper_files/
+paper/*.Rout
diff --git a/DESCRIPTION b/DESCRIPTION
@@ -1,6 +1,6 @@
 Package: wdpar
 Type: Package
-Version: 1.3.3
+Version: 1.3.3.1
 Title: Interface to the World Database on Protected Areas
 Description: Fetch and clean data from the World Database on Protected
     Areas (WDPA) and the World Database on Other Effective Area-Based
@@ -18,8 +18,8 @@ Imports:
     curl (>= 3.2),
     httr (>= 1.3.1),
     countrycode (>= 1.1.0),
-    wdman (>= 0.2.4),
-    RSelenium (>= 1.7.4),
+    withr (>= 2.5.0),
+    webdriver (>= 1.0.6),
     xml2 (>= 1.2.0),
     cli (>= 1.0.1),
     lwgeom (>= 0.2-1),
@@ -45,7 +45,7 @@ Language: en-US
 URL: https://prioritizr.github.io/wdpar/, https://github.com/prioritizr/wdpar
 BugReports: https://github.com/prioritizr/wdpar/issues
 VignetteBuilder: knitr
-RoxygenNote: 7.2.0
+RoxygenNote: 7.2.1
 Collate:
     'internal.R'
     'package.R'

diff --git a/Makefile b/Makefile
@@ -51,7 +51,7 @@ build:
 	cp -R doc inst/
 
 install:
-	R --slave -e "devtools::install_local('../wdpar', force = TRUE)"
+	R --slave -e "devtools::install_local(force = TRUE)"
 
 spellcheck:
 	echo "\n===== SPELL CHECK =====\n" > spell.log 2>&1
@@ -60,13 +60,17 @@ spellcheck:
 urlcheck:
 	R --slave -e "devtools::document();urlchecker::url_check()"
 
+purl_vigns:
+	R --slave -e "lapply(dir('vignettes', '^.*\\\\.Rmd$$'), function(x) knitr::purl(file.path('vignettes', x), gsub('.Rmd', '.R', x, fixed = TRUE)))"
+	rm -f Rplots.pdf
+
 examples:
 	R --slave -e "devtools::run_examples(run_donttest = TRUE, run_dontrun = TRUE);warnings()"  >> examples.log
 	rm -f Rplots.pdf
 
-paper: paper/paper.pdf
+paper: paper/paper.pdf paper/paper.docx
 
-paper/paper.pdf: paper/paper.Rmd paper/paper.bib
+paper/paper.pdf: paper/paper.Rmd paper/paper.bib paper/carbon.png
 	rm -f paper/paper.pdf
 	rm -f paper/paper.md
 	R --slave -e "rmarkdown::render('paper/paper.Rmd', output_file = 'paper.pdf')"
@@ -80,9 +84,16 @@ paper/paper.pdf: paper/paper.Rmd paper/paper.bib
     --env JOURNAL=joss \
     openjournals/inara
 	rm -f paper/paper.jats
+
+paper/paper.docx: paper/paper.pdf
 	R --slave -e "rmarkdown::render('paper/paper.Rmd', output_file = 'paper.docx', output_format = 'word_document')"
 
+paper_case_study:
+	R CMD BATCH --no-restore --no-save paper/case-study.R
+	mv case-study.Rout paper/case-study.Rout
+	rm -f Rplots.pdf
+
 wdpa_global: install
 	R CMD BATCH --no-restore --no-save inst/scripts/global-example-script.R
 
-.PHONY: initc vigns clean data docs readme site test check checkwb build  install man spellcheck examples wdpa_global paper
+.PHONY: initc vigns clean data docs readme site test check checkwb build  install man spellcheck examples wdpa_global paper purl_vigns paper_case_study
diff --git a/NEWS.md b/NEWS.md
@@ -1,3 +1,14 @@
+# wdpar 1.3.3.1
+
+- Update paper for JOSS submission (#53, #54, #62).
+- Update vignette with new section on local scale analyses (#53).
+- Update `wdpa_fetch()` function to use the webdriver package for obtaining data
+  (replacing Rselenium package as a dependency) (#63).
+- Update `st_repair_geometry()` to be more robust.
+- Fix failing tests for `st_repair_geometry()` function.
+- Update documentation for `wdpa_clean()` function.
+- Fix broken URL in vignette.
+
 # wdpar 1.3.3
 
 - CRAN release.

diff --git a/R/st_repair_geometry.R b/R/st_repair_geometry.R
@@ -67,6 +67,23 @@ st_repair_geometry <- function(x, geometry_precision = 1500) {
   # extract polygons and points (if needed)
   x2 <- extract_polygons_and_points(x2)
 
+  # dissolve by repair id
+  if (!identical(anyDuplicated(x2[["_repair_id"]]), 0L)) {
+    x2 <- split(x2, x2[["_repair_id"]])
+    x2_df <- tibble::tibble(`_repair_id` = as.integer(names(x2)))
+    x2 <- lapply(x2, sf::st_geometry)
+    x2 <- lapply(x2, sf::st_union)
+    x2 <- do.call(c, x2)
+    x_df <- match(x2_df[["_repair_id"]], x[["_repair_id"]])
+    x_df <- sf::st_drop_geometry(x)[x_df, , drop = FALSE]
+    x_df <- x_df[, setdiff(names(x_df), "_repair_id"), drop = FALSE]
+    x2_df <- tibble::as_tibble(cbind(x2_df, x_df))
+    x2_df <- x2_df[, names(sf::st_drop_geometry(x)), , drop = FALSE]
+    x2_df$geometry <- x2
+    x2 <- sf::st_sf(x2_df)
+    rm(x_df, x2_df)
+  }
+
   # detect if any invalid geometries persist
   ## subset repaired polygons
   x_sub <- x[match(x2[["_repair_id"]], x[["_repair_id"]]), , drop = FALSE]

diff --git a/R/wdpa_clean.R b/R/wdpa_clean.R
@@ -67,8 +67,6 @@ NULL
 #'
 #'   \enumerate{
 #'
-#'   \item Repair invalid geometry (using [sf::st_make_valid()]).
-#'
 #'   \item Exclude protected areas according to their status (i.e.
 #'     `"STATUS"` field). Specifically, protected areas that have
 #'     a status not specified in the argument to `retain_status` are excluded.

diff --git a/R/wdpa_url.R b/R/wdpa_url.R
@@ -60,33 +60,31 @@ wdpa_url <- function(x, wait = FALSE, page_wait = 2) {
     assertthat::is.flag(wait),
     assertthat::is.count(page_wait),
     assertthat::noNA(page_wait),
-    is_online())
+    is_online(),
+    has_phantomjs(silent = FALSE))
   # declare hidden function
   try_and_find_url <- function(x) {
     ## initialize web driver
     result <- suppressMessages(tryCatch({
       ## initialize URL
       url <- character(0)
-      ## specify port
-      port <- as.integer(ceiling(sample(14415:14935, 1)))
+      ## initialize driver
+      pjs <- start_phantomjs()
+      rd <- webdriver::Session$new(port = pjs$port)
       ## navigate to download web page
-      pjs <- wdman::phantomjs(port = port, verbose = FALSE)
-      rd <- RSelenium::remoteDriver(port = port, browserName = "phantomjs")
-      rd$open(silent = TRUE)
-      rd$maxWindowSize()
-      rd$navigate(paste0("https://protectedplanet.net/country/", x))
+      rd$go(paste0("https://protectedplanet.net/country/", x))
       Sys.sleep(page_wait) # wait for page to load
-      elem <- rd$findElement(using = "css", ".download__trigger")
-      elem$clickElement()
+      elem <- rd$findElement(css = ".download__trigger")
+      elem$click()
       Sys.sleep(page_wait) # wait for page to load
-      elem <- rd$findElement(using = "css", "li:nth-child(2) .popup__link")
-      elem$clickElement()
+      elem <- rd$findElement(css = "li:nth-child(2) .popup__link")
+      elem$click()
       Sys.sleep(page_wait) # wait for dialog to open
-      elem <- rd$findElement(using = "css", ".modal__link-button")
-      elem$clickElement()
+      elem <- rd$findElement(css = ".modal__link-button")
+      elem$click()
       Sys.sleep(page_wait) # wait for for dialog to open
       ## extract html for modal
-      src <- xml2::read_html(rd$getPageSource()[[1]][[1]], encoding = "UTF-8")
+      src <- xml2::read_html(rd$getSource()[[1]][[1]], encoding = "UTF-8")
       divs <- xml2::xml_find_all(src, ".//div")
       divs <- divs[which(xml2::xml_attr(divs, "class") == "modal__content")]
       ## parse download link
@@ -95,13 +93,10 @@ wdpa_url <- function(x, wait = FALSE, page_wait = 2) {
     },
     finally = {
       ## clean up web driver
-      try(ph <- pjs$process)
-      try(rd$close(), silent = TRUE)
-      try(rd$close(), silent = TRUE)
-      try(pjs$stop(), silent = TRUE)
-      try(pjs$stop(), silent = TRUE)
-      try(ph$kill_tree(), silent = TRUE)
-      try(ph$kill_tree(), silent = TRUE)
+      try(rd$delete(), silent = TRUE)
+      try(rd$delete(), silent = TRUE)
+      try(stop_phantomjs(pjs), silent = TRUE)
+      try(stop_phantomjs(pjs), silent = TRUE)
     }))
     ## prepare output
     if (length(url) == 0)
@@ -137,3 +132,38 @@ wdpa_url <- function(x, wait = FALSE, page_wait = 2) {
   # return url
   return(out)
 }
+
+start_phantomjs <- function() {
+  # initialize phantomjs
+  if (
+    identical(.Platform$OS.type, "unix") &&
+    identical(Sys.getenv("OPENSSL_CONF"), "")
+  ) {
+    withr::with_envvar(
+      list("OPENSSL_CONF"= "/etc/ssl"),
+      pjs <- webdriver::run_phantomjs()
+    )
+  } else {
+    pjs <- webdriver::run_phantomjs()
+  }
+  # return object
+  pjs
+}
+
+stop_phantomjs <- function(pjs) {
+  try(pjs$process$kill(), silent = TRUE)
+  try(pjs$process$kill(), silent = TRUE)
+}
+
+has_phantomjs <- function(silent = TRUE) {
+  assertthat::assert_that(
+    assertthat::is.flag(silent),
+    assertthat::noNA(silent)
+  )
+  pjs <- try(start_phantomjs, silent = TRUE)
+  on.exit(stop_phantomjs(pjs))
+  if (inherits(pjs, "try-error") && !isTRUE(silent)) {
+    stop(pjs)
+  }
+  !inherits(pjs, "try-error")
+}
diff --git a/README.md b/README.md
@@ -182,21 +182,21 @@ head(mlt_pa_data)
     ## CRS:           +proj=cea +lon_0=0 +lat_ts=30 +x_0=0 +y_0=0 +datum=WGS84 +ellps=WGS84 +units=m +no_defs
     ## Precision:     1500 
     ## # A tibble: 6 × 33
-    ##      WDPAID WDPA_PID  PA_DEF NAME  ORIG_NAME DESIG DESIG_ENG DESIG_TYPE IUCN_CAT
-    ##       <dbl> <chr>     <chr>  <chr> <chr>     <chr> <chr>     <chr>      <chr>   
-    ## 1    194420 194420    PA     Filf… Filfla    Rise… Nature R… National   Ia      
-    ## 2    194425 194425    PA     Il-G… Il-Gżejj… Rise… Nature R… National   Ia      
-    ## 3 555588631 555588631 PA     Il-M… Il-Majji… Park… National… National   II      
-    ## 4    174757 174757    PA     Il-Ġ… Il-Ġonna… List… List of … National   III     
-    ## 5    174758 174758    PA     Bidn… Bidnija,… List… List of … National   III     
-    ## 6    194415 194415    PA     Il-Ġ… Il-Ġonna… List… List of … National   III     
-    ## # … with 24 more variables: INT_CRIT <chr>, MARINE <chr>, REP_M_AREA <dbl>,
-    ## #   GIS_M_AREA <dbl>, REP_AREA <dbl>, GIS_AREA <dbl>, NO_TAKE <chr>,
-    ## #   NO_TK_AREA <dbl>, STATUS <chr>, STATUS_YR <dbl>, GOV_TYPE <chr>,
-    ## #   OWN_TYPE <chr>, MANG_AUTH <chr>, MANG_PLAN <chr>, VERIF <chr>,
-    ## #   METADATAID <int>, SUB_LOC <chr>, PARENT_ISO <chr>, ISO3 <chr>,
-    ## #   SUPP_INFO <chr>, CONS_OBJ <chr>, GEOMETRY_TYPE <chr>, AREA_KM2 <dbl>,
-    ## #   geometry <MULTIPOLYGON [m]>
+    ##      WDPAID WDPA_PID  PA_DEF NAME  ORIG_…¹ DESIG DESIG…² DESIG…³ IUCN_…⁴ INT_C…⁵
+    ##       <dbl> <chr>     <chr>  <chr> <chr>   <chr> <chr>   <chr>   <chr>   <chr>  
+    ## 1    194420 194420    PA     Filf… Filfla  Rise… Nature… Nation… Ia      Not Ap…
+    ## 2    194425 194425    PA     Il-G… Il-Gże… Rise… Nature… Nation… Ia      Not Ap…
+    ## 3 555588631 555588631 PA     Il-M… Il-Maj… Park… Nation… Nation… II      Not Ap…
+    ## 4    174757 174757    PA     Il-Ġ… Il-Ġon… List… List o… Nation… III     Not Ap…
+    ## 5    174758 174758    PA     Bidn… Bidnij… List… List o… Nation… III     Not Ap…
+    ## 6    194415 194415    PA     Il-Ġ… Il-Ġon… List… List o… Nation… III     Not Ap…
+    ## # … with 23 more variables: MARINE <chr>, REP_M_AREA <dbl>, GIS_M_AREA <dbl>,
+    ## #   REP_AREA <dbl>, GIS_AREA <dbl>, NO_TAKE <chr>, NO_TK_AREA <dbl>,
+    ## #   STATUS <chr>, STATUS_YR <dbl>, GOV_TYPE <chr>, OWN_TYPE <chr>,
+    ## #   MANG_AUTH <chr>, MANG_PLAN <chr>, VERIF <chr>, METADATAID <int>,
+    ## #   SUB_LOC <chr>, PARENT_ISO <chr>, ISO3 <chr>, SUPP_INFO <chr>,
+    ## #   CONS_OBJ <chr>, GEOMETRY_TYPE <chr>, AREA_KM2 <dbl>,
+    ## #   geometry <MULTIPOLYGON [m]>, and abbreviated variable names ¹ORIG_NAME, …
 
 Finally, after cleaning the data, let’s plot a map showing Malta’s
 protected areas and color each area according to its management category
@@ -247,7 +247,7 @@ To cite the latest official version, please use:
 To cite the latest development version, please use:
 
 > Hanson JO (2022). wdpar: Interface to the World Database on Protected
-> Areas. R package version 1.3.3. Available at
+> Areas. R package version 1.3.3.1. Available at
 > <https://github.com/prioritizr/wdpar>.
 
 To cite the World Database on Protected Areas (WDPA), please use:

diff --git a/docs/404.html b/docs/404.html