Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getting seabed_substrate_1m layer from geology_seabed_substrate_maps fails in EMODnetWFS 2.0.0 #23

Closed
JanJaapPoos opened this issue Oct 22, 2021 · 9 comments

Comments

@JanJaapPoos
Copy link

JanJaapPoos commented Oct 22, 2021

Just installed EMODnetWFS and getting the seabed_substrate_1m layer from geology_seabed_substrate_maps now fails. I think that it also failed for 1.0.0, but I am not sure. This is on windows, R3.6.1.

wfs <- emodnet_init_wfs_client(service="geology_seabed_substrate_maps")
Loading ISO 19139 XML schemas...
Loading ISO 19115 codelists...
√ WFS client created succesfully
i Service: 'https://drive.emodnet-geology.eu/geoserver/gtk/wfs'
i Version: '2.0.0'
test <- emodnet_get_layers(wfs = wfs, layers = "seabed_substrate_1m", reduce_layers = T)
Space required after the Public Identifier
SystemLiteral " or ' expected
SYSTEM or PUBLIC, the URI is missing
Entity 'nbsp' not defined
Warning: Download of layer 'seabed_substrate_1m' failed: Error: 1: Space required after the Public Identifier
2: SystemLiteral " or ' expected
3: SYSTEM or PUBLIC, the URI is missing
4: Entity 'nbsp' not defined

The error is thus "Space required after the Public Identifier". Some digging in emodnet_get_layers() reveals that this comes from a call to purrr::map2

debug: out <- purrr::map2(.x = layers, .y = cql_filter, ~ews_get_layer(.x,
wfs, cql_filter = .y), wfs, suppress_warnings) %>% stats::setNames(layers)

Space required after the Public Identifier
SystemLiteral " or ' expected
SYSTEM or PUBLIC, the URI is missing
Entity 'nbsp' not defined
Warning: Download of layer 'seabed_substrate_1m' failed: Error: 1: Space required after the Public Identifier
2: SystemLiteral " or ' expected
3: SYSTEM or PUBLIC, the URI is missing
4: Entity 'nbsp' not defined

at this time layers is a single string "seabed_substrate_1m" and cql_filter is NA. The version of purrr is:

packageDescription("purrr")
Package: purrr
Title: Functional Programming Tools
Version: 0.3.4

Any pointers?

To add: getting layers from chemistry_marine_litter as per the example on the github main page works fine.

@annakrystalli
Copy link
Collaborator

Thanks again for reporting this! (and for bearing with this experimental version of the package!). I will have a look at this first thing Monday morning.

@JanJaapPoos
Copy link
Author

Hi Anna, Thanks for looking into this. Note that I added a small addition to the original post: the example for getting layers from chemistry works for me, so this it does not seem to be something general about the setup of my computer.

@annakrystalli
Copy link
Collaborator

Hello there @JanJaapPoos and thanks for your patience. I'm having really strange experiences looking into this error!
So the first time I ran your example, I did get data (it did take some time though! over 8mins).

> wfs <- emodnet_init_wfs_client(service="geology_seabed_substrate_maps")
✓ WFS client created succesfullyService: 'https://drive.emodnet-geology.eu/geoserver/gtk/wfs'Version: '2.0.0'
> test <- emodnet_get_layers(wfs = wfs, layers = "seabed_substrate_1m", reduce_layers = T)
Warning message:
In CPL_read_ogr(dsn, layer, query, as.character(options), quiet,  :
  GDAL Error 1: HTTP error code : 502

Here's the test object I get:

Simple feature collection with 35816 features and 31 fields
Geometry type: MULTIPOLYGON
Dimension:     XY
Bounding box:  xmin: 747069.2 ymin: 347574.3 xmax: 7670427 ymax: 6622268
Projected CRS: ETRS89-extended / LCC Europe
First 10 features:
                   gml_id name id objectid   code country data_holder                            contact   scale original_scale original_grain_size
1   seabed_substrate_1m.1       1        1 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
2   seabed_substrate_1m.2       2        2 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
3   seabed_substrate_1m.3       3        3 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
4   seabed_substrate_1m.4       4        4 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
5   seabed_substrate_1m.5       5        5 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
6   seabed_substrate_1m.6       6        6 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
7   seabed_substrate_1m.7       7        7 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
8   seabed_substrate_1m.8       8        8 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
9   seabed_substrate_1m.9       9        9 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
10 seabed_substrate_1m.10      10       10 BE-003 Belgium       RBINS vera.vanlancker@naturalsciences.be 1000000              0              Folk16
                                                                                           references
1  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
2  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
3  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
4  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
5  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
6  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
7  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
8  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
9  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
10 Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
                                                               comments reclassification method sample_number original_substrate relation folk_16cl
1  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
2  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
3  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
4  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
5  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
6  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
7  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
8  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
9  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
10 Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.                1      1             0                           0         6
              folk_16cl_txt folk_7cl folk_7cl_txt folk_5cl folk_5cl_txt surface_feature shape_length   shape_area conf_rs conf_s conf_d conf_tot
1  6. No data at this level        2      2. Sand        2     2. Sand                   0.012192284 2.149613e-06      NA     NA     NA       NA
2  6. No data at this level        2      2. Sand        2     2. Sand                   0.003482063 4.213989e-08      NA     NA     NA       NA
3  6. No data at this level        2      2. Sand        2     2. Sand                   0.038381086 1.200432e-07      NA     NA     NA       NA
4  6. No data at this level        2      2. Sand        2     2. Sand                   0.083580831 7.006010e-05      NA     NA     NA       NA
5  6. No data at this level        2      2. Sand        2     2. Sand                   0.205756667 1.565305e-04      NA     NA     NA       NA
6  6. No data at this level        2      2. Sand        2     2. Sand                   0.139014139 4.262724e-05      NA     NA     NA       NA
7  6. No data at this level        2      2. Sand        2     2. Sand                   0.065356895 4.309050e-05      NA     NA     NA       NA
8  6. No data at this level        2      2. Sand        2     2. Sand                   0.297000687 3.275952e-05      NA     NA     NA       NA
9  6. No data at this level        2      2. Sand        2     2. Sand                   0.187015766 3.855894e-05      NA     NA     NA       NA
10 6. No data at this level        2      2. Sand        2     2. Sand                   0.120020756 1.209241e-05      NA     NA     NA       NA
                         geometry
1  MULTIPOLYGON (((3550505 276...
2  MULTIPOLYGON (((3541775 276...
3  MULTIPOLYGON (((3541570 276...
4  MULTIPOLYGON (((3539852 277...
5  MULTIPOLYGON (((3536146 277...
6  MULTIPOLYGON (((3481646 278...
7  MULTIPOLYGON (((3489332 279...
8  MULTIPOLYGON (((3528872 278...
9  MULTIPOLYGON (((3520718 279...
10 MULTIPOLYGON (((3495594 279...

As you can see however, I also got a warning which I believe is coming form the interaction between sf and GDAL. Then I decided to run it with reprex() to get a nice markdown print out of the results. I wasn't expecting to get a different result so I didn't call session info to have a record of the state that succeeded (annoyingly).

However, running your example in using reprex() I get exactly the same error you report!

library(EMODnetWFS)
wfs <- emodnet_init_wfs_client(service="geology_seabed_substrate_maps")
#> Loading ISO 19139 XML schemas...
#> Loading ISO 19115 codelists...
#> Space required after the Public Identifier
#> SystemLiteral " or ' expected
#> SYSTEM or PUBLIC, the URI is missing
#> Entity 'nbsp' not defined
#> Error: 1: Space required after the Public Identifier
#> 2: SystemLiteral " or ' expected
#> 3: SYSTEM or PUBLIC, the URI is missing
#> 4: Entity 'nbsp' not defined
t0 <- Sys.time()
test <- emodnet_get_layers(wfs = wfs, layers = "seabed_substrate_1m", reduce_layers = T)
#> Error in emodnet_get_layers(wfs = wfs, layers = "seabed_substrate_1m", : object 'wfs' not found
Sys.time() - t0
#> Time difference of 0.003529072 secs
test
#> Error in eval(expr, envir, enclos): object 'test' not found

Created on 2021-10-25 by the reprex package (v2.0.1)

Oddly, now when I try to rerun on the console, I get the same error, even after restarting a new R session! 🤷‍♀️

Regardless of the odf and inconsistent behaviour described above, it seems the problem must have been introduced when implementing the recent filtering feature as using the version of the package at 11adf76 using install_github("EMODnet/EMODnetWFS", ref="11adf768769df0fac2494d603c210dbb79c7d376")

I get data successfully with no warnings.

library(EMODnetWFS)
wfs <- emodnet_init_wfs_client(service="geology_seabed_substrate_maps")
#> Loading ISO 19139 XML schemas...
#> Loading ISO 19115 codelists...
#> Loading IANA mime types...
#> No encoding supplied: defaulting to UTF-8.
#> ✓ WFS client created succesfully
#> ℹ Service: 'https://drive.emodnet-geology.eu/geoserver/gtk/wfs'
#> ℹ Version: '2.0.0'
t0 <- Sys.time()
test <- emodnet_get_layers(wfs = wfs, layers = "seabed_substrate_1m", reduce_layers = T)
Sys.time() - t0
#> Time difference of 3.476063 mins
test
#> Simple feature collection with 35816 features and 30 fields
#> Geometry type: MULTISURFACE
#> Dimension:     XY
#> Bounding box:  xmin: 747069.2 ymin: 347574.3 xmax: 7670427 ymax: 6622268
#> CRS:           +init=epsg:3034
#> First 10 features:
#>                    gml_id id objectid   code country data_holder
#> 1   seabed_substrate_1m.1  1        1 BE-003 Belgium       RBINS
#> 2   seabed_substrate_1m.2  2        2 BE-003 Belgium       RBINS
#> 3   seabed_substrate_1m.3  3        3 BE-003 Belgium       RBINS
#> 4   seabed_substrate_1m.4  4        4 BE-003 Belgium       RBINS
#> 5   seabed_substrate_1m.5  5        5 BE-003 Belgium       RBINS
#> 6   seabed_substrate_1m.6  6        6 BE-003 Belgium       RBINS
#> 7   seabed_substrate_1m.7  7        7 BE-003 Belgium       RBINS
#> 8   seabed_substrate_1m.8  8        8 BE-003 Belgium       RBINS
#> 9   seabed_substrate_1m.9  9        9 BE-003 Belgium       RBINS
#> 10 seabed_substrate_1m.10 10       10 BE-003 Belgium       RBINS
#>                               contact   scale original_scale
#> 1  [email protected] 1000000              0
#> 2  [email protected] 1000000              0
#> 3  [email protected] 1000000              0
#> 4  [email protected] 1000000              0
#> 5  [email protected] 1000000              0
#> 6  [email protected] 1000000              0
#> 7  [email protected] 1000000              0
#> 8  [email protected] 1000000              0
#> 9  [email protected] 1000000              0
#> 10 [email protected] 1000000              0
#>    original_grain_size
#> 1               Folk16
#> 2               Folk16
#> 3               Folk16
#> 4               Folk16
#> 5               Folk16
#> 6               Folk16
#> 7               Folk16
#> 8               Folk16
#> 9               Folk16
#> 10              Folk16
#>                                                                                            references
#> 1  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#> 2  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#> 3  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#> 4  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#> 5  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#> 6  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#> 7  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#> 8  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#> 9  Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#> 10 Van Lancker, V. (2009). SediCURVE@SEA: a multiparameter sedimentdatabase in support of EIA at sea.
#>                                                                comments
#> 1  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#> 2  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#> 3  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#> 4  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#> 5  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#> 6  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#> 7  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#> 8  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#> 9  Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#> 10 Generalized from 250k scale to 1 M by GTK June 2016/EMODnet Geology.
#>    reclassification method sample_number original_substrate relation folk_16cl
#> 1                 1      1             0                           0         6
#> 2                 1      1             0                           0         6
#> 3                 1      1             0                           0         6
#> 4                 1      1             0                           0         6
#> 5                 1      1             0                           0         6
#> 6                 1      1             0                           0         6
#> 7                 1      1             0                           0         6
#> 8                 1      1             0                           0         6
#> 9                 1      1             0                           0         6
#> 10                1      1             0                           0         6
#>               folk_16cl_txt folk_7cl folk_7cl_txt folk_5cl folk_5cl_txt
#> 1  6. No data at this level        2      2. Sand        2     2. Sand 
#> 2  6. No data at this level        2      2. Sand        2     2. Sand 
#> 3  6. No data at this level        2      2. Sand        2     2. Sand 
#> 4  6. No data at this level        2      2. Sand        2     2. Sand 
#> 5  6. No data at this level        2      2. Sand        2     2. Sand 
#> 6  6. No data at this level        2      2. Sand        2     2. Sand 
#> 7  6. No data at this level        2      2. Sand        2     2. Sand 
#> 8  6. No data at this level        2      2. Sand        2     2. Sand 
#> 9  6. No data at this level        2      2. Sand        2     2. Sand 
#> 10 6. No data at this level        2      2. Sand        2     2. Sand 
#>    surface_feature conf_rs conf_s conf_d conf_tot shape_length   shape_area
#> 1                       NA     NA     NA       NA  0.012192284 2.149613e-06
#> 2                       NA     NA     NA       NA  0.003482063 4.213989e-08
#> 3                       NA     NA     NA       NA  0.038381086 1.200432e-07
#> 4                       NA     NA     NA       NA  0.083580831 7.006010e-05
#> 5                       NA     NA     NA       NA  0.205756667 1.565305e-04
#> 6                       NA     NA     NA       NA  0.139014139 4.262724e-05
#> 7                       NA     NA     NA       NA  0.065356895 4.309050e-05
#> 8                       NA     NA     NA       NA  0.297000687 3.275952e-05
#> 9                       NA     NA     NA       NA  0.187015766 3.855894e-05
#> 10                      NA     NA     NA       NA  0.120020756 1.209241e-05
#>                              geom
#> 1  MULTISURFACE (POLYGON ((355...
#> 2  MULTISURFACE (POLYGON ((354...
#> 3  MULTISURFACE (POLYGON ((354...
#> 4  MULTISURFACE (POLYGON ((353...
#> 5  MULTISURFACE (POLYGON ((353...
#> 6  MULTISURFACE (POLYGON ((348...
#> 7  MULTISURFACE (POLYGON ((348...
#> 8  MULTISURFACE (POLYGON ((352...
#> 9  MULTISURFACE (POLYGON ((352...
#> 10 MULTISURFACE (POLYGON ((349...
devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.1.0 (2021-05-18)
#>  os       macOS Big Sur 10.16         
#>  system   x86_64, darwin17.0          
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_GB.UTF-8                 
#>  ctype    en_GB.UTF-8                 
#>  tz       Europe/London               
#>  date     2021-10-25                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version  date       lib source                             
#>  askpass       1.1      2019-01-13 [1] CRAN (R 4.1.0)                     
#>  assertthat    0.2.1    2019-03-21 [1] CRAN (R 4.1.0)                     
#>  backports     1.2.1    2020-12-09 [1] CRAN (R 4.1.0)                     
#>  cachem        1.0.6    2021-08-19 [1] CRAN (R 4.1.0)                     
#>  callr         3.7.0    2021-04-20 [1] CRAN (R 4.1.0)                     
#>  checkmate     2.0.0    2020-02-06 [1] CRAN (R 4.1.0)                     
#>  class         7.3-19   2021-05-03 [1] CRAN (R 4.1.0)                     
#>  classInt      0.4-3    2020-04-07 [1] CRAN (R 4.1.0)                     
#>  cli           3.0.1    2021-07-17 [1] CRAN (R 4.1.0)                     
#>  crayon        1.4.1    2021-02-08 [1] CRAN (R 4.1.0)                     
#>  curl          4.3.2    2021-06-23 [1] CRAN (R 4.1.0)                     
#>  DBI           1.1.1    2021-01-15 [1] CRAN (R 4.1.0)                     
#>  desc          1.4.0    2021-09-28 [1] CRAN (R 4.1.0)                     
#>  devtools      2.4.2    2021-06-07 [1] CRAN (R 4.1.0)                     
#>  digest        0.6.27   2020-10-24 [1] CRAN (R 4.1.0)                     
#>  dplyr         1.0.7    2021-06-18 [1] CRAN (R 4.1.0)                     
#>  e1071         1.7-9    2021-09-16 [1] CRAN (R 4.1.0)                     
#>  ellipsis      0.3.2    2021-04-29 [1] CRAN (R 4.1.0)                     
#>  EMODnetWFS  * 2.0.0    2021-10-25 [1] Github (EMODnet/EMODnetWFS@11adf76)
#>  evaluate      0.14     2019-05-28 [1] CRAN (R 4.1.0)                     
#>  fansi         0.5.0    2021-05-25 [1] CRAN (R 4.1.0)                     
#>  fastmap       1.1.0    2021-01-25 [1] CRAN (R 4.1.0)                     
#>  fs            1.5.0    2020-07-31 [1] CRAN (R 4.1.0)                     
#>  generics      0.1.0    2020-10-31 [1] CRAN (R 4.1.0)                     
#>  geometa       0.6-4    2021-09-16 [1] CRAN (R 4.1.0)                     
#>  glue          1.4.2    2020-08-27 [1] CRAN (R 4.1.0)                     
#>  highr         0.9      2021-04-16 [1] CRAN (R 4.1.0)                     
#>  htmltools     0.5.2    2021-08-25 [1] CRAN (R 4.1.0)                     
#>  httr          1.4.2    2020-07-20 [1] CRAN (R 4.1.0)                     
#>  jsonlite      1.7.2    2020-12-09 [1] CRAN (R 4.1.0)                     
#>  KernSmooth    2.23-20  2021-05-03 [1] CRAN (R 4.1.0)                     
#>  knitr         1.33     2021-04-24 [1] CRAN (R 4.1.0)                     
#>  lattice       0.20-44  2021-05-02 [1] CRAN (R 4.1.0)                     
#>  lifecycle     1.0.1    2021-09-24 [1] CRAN (R 4.1.0)                     
#>  magrittr      2.0.1    2020-11-17 [1] CRAN (R 4.1.0)                     
#>  memoise       2.0.0    2021-01-26 [1] CRAN (R 4.1.0)                     
#>  openssl       1.4.5    2021-09-02 [1] CRAN (R 4.1.0)                     
#>  ows4R         0.1-5    2020-05-27 [1] CRAN (R 4.1.0)                     
#>  pillar        1.6.4    2021-10-18 [1] CRAN (R 4.1.0)                     
#>  pkgbuild      1.2.0    2020-12-15 [1] CRAN (R 4.1.0)                     
#>  pkgconfig     2.0.3    2019-09-22 [1] CRAN (R 4.1.0)                     
#>  pkgload       1.2.1    2021-04-06 [1] CRAN (R 4.1.0)                     
#>  prettyunits   1.1.1    2020-01-24 [1] CRAN (R 4.1.0)                     
#>  processx      3.5.2    2021-04-30 [1] CRAN (R 4.1.0)                     
#>  proxy         0.4-26   2021-06-07 [1] CRAN (R 4.1.0)                     
#>  ps            1.6.0    2021-02-28 [1] CRAN (R 4.1.0)                     
#>  purrr         0.3.4    2020-04-17 [1] CRAN (R 4.1.0)                     
#>  R6            2.5.1    2021-08-19 [1] CRAN (R 4.1.0)                     
#>  Rcpp          1.0.7    2021-07-07 [1] CRAN (R 4.1.0)                     
#>  remotes       2.4.0    2021-06-02 [1] CRAN (R 4.1.0)                     
#>  reprex        2.0.1    2021-08-05 [1] CRAN (R 4.1.0)                     
#>  rgdal         1.5-27   2021-09-16 [1] CRAN (R 4.1.0)                     
#>  rlang         0.4.12   2021-10-18 [1] CRAN (R 4.1.0)                     
#>  rmarkdown     2.10     2021-08-06 [1] CRAN (R 4.1.0)                     
#>  rprojroot     2.0.2    2020-11-15 [1] CRAN (R 4.1.0)                     
#>  rstudioapi    0.13     2020-11-12 [1] CRAN (R 4.1.0)                     
#>  sessioninfo   1.1.1    2018-11-05 [1] CRAN (R 4.1.0)                     
#>  sf            1.0-3    2021-10-07 [1] CRAN (R 4.1.0)                     
#>  sp            1.4-5    2021-01-10 [1] CRAN (R 4.1.0)                     
#>  stringi       1.7.5    2021-10-04 [1] CRAN (R 4.1.0)                     
#>  stringr       1.4.0    2019-02-10 [1] CRAN (R 4.1.0)                     
#>  styler        1.5.1    2021-07-13 [1] CRAN (R 4.1.0)                     
#>  testthat      3.0.4    2021-07-01 [1] CRAN (R 4.1.0)                     
#>  tibble        3.1.5    2021-09-30 [1] CRAN (R 4.1.0)                     
#>  tidyr         1.1.4    2021-09-27 [1] CRAN (R 4.1.0)                     
#>  tidyselect    1.1.1    2021-04-30 [1] CRAN (R 4.1.0)                     
#>  units         0.7-2    2021-06-08 [1] CRAN (R 4.1.0)                     
#>  usethis       2.1.0    2021-10-16 [1] CRAN (R 4.1.0)                     
#>  utf8          1.2.2    2021-07-24 [1] CRAN (R 4.1.0)                     
#>  vctrs         0.3.8    2021-04-29 [1] CRAN (R 4.1.0)                     
#>  withr         2.4.2    2021-04-18 [1] CRAN (R 4.1.0)                     
#>  xfun          0.25     2021-08-06 [1] CRAN (R 4.1.0)                     
#>  XML           3.99-0.8 2021-09-17 [1] CRAN (R 4.1.0)                     
#>  yaml          2.2.1    2020-02-01 [1] CRAN (R 4.1.0)                     
#> 
#> [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library

Created on 2021-10-25 by the reprex package (v2.0.1)

So I still haven't figured out and fixed the problem. Just wanted to report back what I've found so far (to record it in the issue too).

Will update here with further progress.

@JanJaapPoos
Copy link
Author

Hi @annakrystalli,

Just to confirm that I also received the "HTTP error code : 502" once when I was trying to get data. It did not seem to be consistent, so I was thinking it was some unfortunate mishap in my connection.

I am glad that you managed to reproduce the error. I will use the 11adf76 version as you suggest.

Thanks for your efforts.

@annakrystalli
Copy link
Collaborator

annakrystalli commented Oct 26, 2021

Hi @JanJaapPoos

So more confusion and inconsistency. I've now been able to download data twice last night and twice this morning from the servers using the latest version of the package (ie after running install_github("EMODnet/EMODnetWFS")) without any warning, so it doesn't seem that the filtering feature was the issue.

The consistent error about Entity 'nbsp' not defined seems to be an xml parsing error.

I also noticed that the first time I managed to download data, the object seemed to have a different CRS (Projected CRS: ETRS89-extended / LCC Europe) compared to all the other times +init=epsg:3034, so I wonder if there was some work going on server side the last few days?

@JanJaapPoos, Can you please try again (with the latest package EMODnetWFS version) and let me know the result? If you could include your session info and also run sf::sf_extSoftVersion() too for me?

@salvafern do you have any ideas? It feels like we need to work on better communications with the server maintainers somehow. I'll open another issue for that.

@salvafern
Copy link
Collaborator

Yes it is likely that the server maintainers are changing things. I'll follow up in #24 .

As an intermediate solution: What do you think about adding handlers for the errors? e.g. tryCatch() etc

@annakrystalli
Copy link
Collaborator

So we already do use tryCatch but perhaps it's not working well enough (the code might be failing in unexpected places). Sadly I can't reproduce the errors now but will see if I can figure something out from console output I've still got and perhaps see if I can put together some more tests.

@JanJaapPoos
Copy link
Author

JanJaapPoos commented Oct 26, 2021

Hi @annakrystalli ,

You're right, works fine for me too now. Some relevant info:

crs according to st_crs()

st_crs(test)
Coordinate Reference System:
User input: +init=epsg:3034
wkt:
PROJCRS["ETRS89-extended / LCC Europe",
BASEGEOGCRS["ETRS89",
DATUM["European Terrestrial Reference System 1989",
ELLIPSOID["GRS 1980",6378137,298.257222101,
LENGTHUNIT["metre",1]]],
PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
ID["EPSG",4258]],
CONVERSION["Europe Conformal 2001",
METHOD["Lambert Conic Conformal (2SP)",
ID["EPSG",9802]],
PARAMETER["Latitude of false origin",52,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8821]],
PARAMETER["Longitude of false origin",10,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8822]],
PARAMETER["Latitude of 1st standard parallel",35,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8823]],
PARAMETER["Latitude of 2nd standard parallel",65,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8824]],
PARAMETER["Easting at false origin",4000000,
LENGTHUNIT["metre",1],
ID["EPSG",8826]],
PARAMETER["Northing at false origin",2800000,
LENGTHUNIT["metre",1],
ID["EPSG",8827]],
ID["EPSG",19985]],
CS[Cartesian,2],
AXIS["(E)",east,
ORDER[1],
LENGTHUNIT["metre",1,
ID["EPSG",9001]]],
AXIS["(N)",north,
ORDER[2],
LENGTHUNIT["metre",1,
ID["EPSG",9001]]],
USAGE[
SCOPE["unknown"],
AREA["Europe - European Union (EU) countries and candidates. Europe - onshore and offshore: Albania; Andorra; Austria; Belgium; Bosnia and Herzegovina; Bulgaria; Croatia; Cyprus; Czechia; Denmark; Estonia; Faroe Islands; Finland; France; Germany; Gibraltar; Greece; Hungary; Iceland; Ireland; Italy; Kosovo; Latvia; Liechtenstein; Lithuania; Luxembourg; Malta; Monaco; Montenegro; Netherlands; North Macedonia; Norway including Svalbard and Jan Mayen; Poland; Portugal including Madeira and Azores; Romania; San Marino; Serbia; Slovakia; Slovenia; Spain including Canary Islands; Sweden; Switzerland; Turkey; United Kingdom (UK) including Channel Islands and Isle of Man; Vatican City State."],
BBOX[24.6,-35.58,84.17,44.83]]]

Sessioninfo:

sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C LC_TIME=English_United Kingdom.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] EMODnetWFS_2.0.0 sf_1.0-2

loaded via a namespace (and not attached):
[1] Rcpp_1.0.2 pillar_1.6.2 compiler_3.6.1 ows4R_0.1-5 class_7.3-15 tools_3.6.1 jsonlite_1.6 lifecycle_1.0.0 tibble_3.1.1 checkmate_2.0.0 lattice_0.20-41 pkgconfig_2.0.2
[13] rlang_0.4.11 rstudioapi_0.13 cli_1.1.0 DBI_1.0.0 curl_4.0 rgdal_1.4-6 e1071_1.7-2 geometa_0.6-3 dplyr_1.0.7 httr_1.4.2 fs_1.3.1 generics_0.0.2
[25] vctrs_0.3.8 askpass_1.1 classInt_0.4-3 grid_3.6.1 tidyselect_1.1.0 glue_1.4.2 R6_2.4.0 fansi_0.4.0 XML_3.99-0.3 sp_1.3-1 tidyr_1.1.3 purrr_0.3.4
[37] magrittr_2.0.1 usethis_2.0.1 backports_1.1.4 ellipsis_0.3.2 units_0.6-3 assertthat_0.2.1 utf8_1.1.4 KernSmooth_2.23-15 openssl_1.4.4 crayon_1.4.1

sf::sf_extSoftVersion()
GEOS GDAL proj.4 GDAL_with_GEOS USE_PROJ_H PROJ
"3.9.0" "3.2.1" "7.2.1" "true" "true" "7.2.1"

@annakrystalli
Copy link
Collaborator

Thanks for the info @JanJaapPoos and so happy it's all working for you again. We really appreciate you taking the time to report errors to us and help us make the package (and underlying services better).

And while the crs is technically the same, it's the expression that seemed to be different. I know there's been big changes in the geospatial software landscape in terms of expressing CRSs as a result of versions of the PROJ library > 5 & GDAL > 3 (see for example discussions of handling changes in package rgdal and here in sf. Indeed the +init CRS specification is deprecated so perhaps they are looking into upgrading CRS representation on the server. In any case, the sf objects return should have a valid WKT specification of the CRS but something to be aware of.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants