Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

weather_dl() throws "Error: guess_max must be a positive integer" #119

Closed
KevCaz opened this issue Apr 29, 2021 · 6 comments
Closed

weather_dl() throws "Error: guess_max must be a positive integer" #119

KevCaz opened this issue Apr 29, 2021 · 6 comments

Comments

@KevCaz
Copy link
Member

KevCaz commented Apr 29, 2021

Hi @steffilazerte,

Expected Behavior

kam <- weather_dl(station_ids = 51423, 
                     start = "2016-01-01", end = "2016-02-15")

Should return a data frame with desired data.

Current Behavior

R> kam <- weather_dl(station_ids = 51423, 
                     start = "2016-01-01", end = "2016-02-15")   
Error: `guess_max` must be a positive integer

Steps to Reproduce (for bugs)

  1. Install latest version
  2. run first example from weather.R

Possible Solution

A quick investigation of mine showed that

weathercan/R/weather.R

Line 659 in fe13093

i <- stringr::str_which(split, "If Local Standard Time|Legend")[1] - 1

now returns a NA which I think explains the error return by read_tsv(). When I changed this, then

weathercan/R/weather.R

Lines 661 to 663 in fe13093

r <- httr::content(html, as = "text",
type = "text/csv",
encoding = encoding) %>%

becomes a one colum data frame (instead of a 2 column one), so there might be more to this. Not sure why, I may investigate this further. Anyway, thanks for your job on this very useful package.

Context

Running some code that was previously working.

Your Environment

See below for my config, note that I was also able to reproduce this on another machine (MacOS 11.2.3 R 4.0.4).

``` sessionInfo() R version 4.0.4 (2021-02-15) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Debian GNU/Linux bullseye/sid

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.13.so

locale:
[1] LC_CTYPE=en_CA.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_CA.UTF-8 LC_COLLATE=en_CA.UTF-8
[5] LC_MONETARY=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8
[7] LC_PAPER=en_CA.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods
[7] base

other attached packages:
[1] vcr_0.6.0 weathercan_0.5.0.9000
[3] rhub_1.1.1 pkgdown_1.6.1
[5] microbenchmark_1.4-7 goodpractice_1.0.2
[7] testthat_3.0.1 devtools_2.3.2
[9] usethis_2.0.0

loaded via a namespace (and not attached):
[1] httr_1.4.2 pkgload_1.1.0 tidyr_1.1.3
[4] jsonlite_1.7.2 assertthat_0.2.1 xmlparsedata_1.0.5
[7] triebeard_0.3.0 urltools_1.7.3 yaml_2.2.1
[10] remotes_2.2.0 sessioninfo_1.1.1 pillar_1.6.0
[13] glue_1.4.2 uuid_0.1-4 digest_0.6.27
[16] rvest_1.0.0 clisymbols_1.2.0 pkgconfig_2.0.3
[19] rcmdcheck_1.3.3 httpcode_0.3.0 purrr_0.3.4
[22] processx_3.4.5 whisker_0.4 webmockr_0.7.4
[25] tibble_3.1.1 generics_0.1.0 ellipsis_0.3.1
[28] cachem_1.0.4 withr_2.4.1 lazyeval_0.2.2
[31] cli_2.5.0 magrittr_2.0.1 crayon_1.4.1
[34] memoise_2.0.0 ps_1.6.0 fs_1.5.0
[37] fansi_0.4.2 xml2_1.3.2 parsedate_1.2.1
[40] pkgbuild_1.2.0 praise_1.0.0 tools_4.0.4
[43] data.table_1.14.0 prettyunits_1.1.1 hms_1.0.0
[46] cyclocomp_1.1.0 lifecycle_1.0.0 stringr_1.4.0
[49] xopen_1.0.0 targets_0.4.1 callr_3.5.1
[52] rex_1.2.0 compiler_4.0.4 covr_3.5.1
[55] rlang_0.4.10 rstudioapi_0.13 rappdirs_0.3.3
[58] igraph_1.2.6 base64enc_0.1-3 codetools_0.2-18
[61] DBI_1.1.1 rematch_1.0.1 curl_4.3
[64] fauxpas_0.5.0 R6_2.5.0 lubridate_1.7.10
[67] dplyr_1.0.5 fastmap_1.1.0 utf8_1.1.4
[70] rprojroot_2.0.2 lintr_2.0.1 readr_1.4.0
[73] desc_1.2.0 stringi_1.5.3 crul_1.1.0
[76] whoami_1.3.0 Rcpp_1.0.6 vctrs_0.3.7
[79] tidyselect_1.1.0

R> devtools::session_info()
─ Session info ────────────────────────────────────────────────
setting value
version R version 4.0.4 (2021-02-15)
os Debian GNU/Linux bullseye/sid
system x86_64, linux-gnu
ui X11
language en_CA:en
collate en_CA.UTF-8
ctype en_CA.UTF-8
tz America/Toronto
date 2021-04-29

─ Packages ────────────────────────────────────────────────────
! package * version date lib
assertthat 0.2.1 2019-03-21 [3]
base64enc 0.1-3 2015-07-28 [3]
cachem 1.0.4 2021-02-13 [1]
callr 3.5.1 2020-10-13 [3]
cli 2.5.0 2021-04-26 [1]
clisymbols 1.2.0 2017-05-21 [3]
codetools 0.2-18 2020-11-04 [4]
covr 3.5.1 2020-09-16 [3]
crayon 1.4.1 2021-02-08 [1]
crul 1.1.0 2021-04-06 [1]
curl 4.3 2019-12-02 [3]
cyclocomp 1.1.0 2016-09-10 [1]
data.table 1.14.0 2021-02-21 [1]
DBI 1.1.1 2021-01-15 [3]
desc 1.2.0 2018-05-01 [3]
devtools * 2.3.2 2020-09-18 [3]
digest 0.6.27 2020-10-24 [3]
dplyr 1.0.5 2021-03-05 [1]
ellipsis 0.3.1 2020-05-15 [3]
fansi 0.4.2 2021-01-15 [3]
fastmap 1.1.0 2021-01-25 [3]
fauxpas 0.5.0 2020-04-13 [3]
fs 1.5.0 2020-07-31 [1]
generics 0.1.0 2020-10-31 [3]
glue 1.4.2 2020-08-27 [3]
goodpractice * 1.0.2 2018-05-02 [1]
hms 1.0.0 2021-01-13 [3]
httpcode 0.3.0 2020-04-10 [3]
httr 1.4.2 2020-07-20 [3]
igraph 1.2.6 2020-10-06 [3]
jsonlite 1.7.2 2020-12-09 [1]
lazyeval 0.2.2 2019-03-15 [3]
lifecycle 1.0.0 2021-02-15 [1]
lintr 2.0.1 2020-02-19 [1]
lubridate 1.7.10 2021-02-26 [1]
magrittr 2.0.1 2020-11-17 [3]
memoise 2.0.0 2021-01-26 [3]
microbenchmark * 1.4-7 2019-09-24 [1]
parsedate 1.2.1 2021-04-20 [1]
pillar 1.6.0 2021-04-13 [1]
pkgbuild 1.2.0 2020-12-15 [3]
pkgconfig 2.0.3 2019-09-22 [3]
pkgdown * 1.6.1 2020-09-12 [1]
pkgload 1.1.0 2020-05-29 [3]
praise 1.0.0 2015-08-11 [3]
prettyunits 1.1.1 2020-01-24 [3]
processx 3.4.5 2020-11-30 [3]
ps 1.6.0 2021-02-28 [1]
purrr 0.3.4 2020-04-17 [3]
R6 2.5.0 2020-10-28 [3]
rappdirs 0.3.3 2021-01-31 [3]
rcmdcheck 1.3.3 2019-05-07 [3]
Rcpp 1.0.6 2021-01-15 [3]
readr 1.4.0 2020-10-05 [3]
rematch 1.0.1 2016-04-21 [3]
remotes 2.2.0 2020-07-21 [3]
rex 1.2.0 2020-04-21 [3]
rhub * 1.1.1 2019-04-08 [1]
rlang 0.4.10 2020-12-30 [3]
rprojroot 2.0.2 2020-11-15 [1]
rstudioapi 0.13 2020-11-12 [3]
rvest 1.0.0 2021-03-09 [1]
sessioninfo 1.1.1 2018-11-05 [3]
stringi 1.5.3 2020-09-09 [1]
stringr 1.4.0 2019-02-10 [3]
targets 0.4.1 2021-04-22 [1]
testthat * 3.0.1 2020-12-17 [3]
tibble 3.1.1 2021-04-18 [1]
tidyr 1.1.3 2021-03-03 [1]
tidyselect 1.1.0 2020-05-11 [3]
triebeard 0.3.0 2016-08-04 [3]
urltools 1.7.3 2019-04-14 [3]
usethis * 2.0.0 2020-12-10 [3]
utf8 1.1.4 2018-05-24 [3]
uuid 0.1-4 2020-02-26 [3]
vcr * 0.6.0 2020-12-12 [3]
vctrs 0.3.7 2021-03-29 [1]
P weathercan * 0.5.0.9000 2021-03-03 [?]
webmockr 0.7.4 2020-12-09 [3]
whisker 0.4 2019-08-28 [3]
whoami 1.3.0 2019-03-19 [1]
withr 2.4.1 2021-01-26 [3]
xml2 1.3.2 2020-04-23 [3]
xmlparsedata 1.0.5 2021-03-06 [1]
xopen 1.0.0 2018-09-17 [3]
yaml 2.2.1 2020-02-01 [3]
source
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.3)
CRAN (R 4.0.3)
CRAN (R 4.0.4)
CRAN (R 4.0.0)
CRAN (R 4.0.3)
CRAN (R 4.0.4)
CRAN (R 4.0.4)
Github (ropensci/crul@a22b191)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.4)
CRAN (R 4.0.3)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.3)
CRAN (R 4.0.4)
CRAN (R 4.0.0)
CRAN (R 4.0.3)
CRAN (R 4.0.3)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.3)
CRAN (R 4.0.2)
CRAN (R 4.0.2)
CRAN (R 4.0.3)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.3)
CRAN (R 4.0.3)
CRAN (R 4.0.0)
CRAN (R 4.0.4)
CRAN (R 4.0.2)
CRAN (R 4.0.4)
CRAN (R 4.0.3)
CRAN (R 4.0.3)
CRAN (R 4.0.2)
CRAN (R 4.0.4)
CRAN (R 4.0.4)
CRAN (R 4.0.3)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.1)
CRAN (R 4.0.0)
CRAN (R 4.0.0)
CRAN (R 4.0.3)
CRAN (R 4.0.4)
CRAN (R 4.0.0)
CRAN (R 4.0.3)
CRAN (R 4.0.3)
CRAN (R 4.0.0)
CRAN (R 4.0.3)
CRAN (R 4.0.3)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.3)
CRAN (R 4.0.3)
CRAN (R 4.0.3)
CRAN (R 4.0.4)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.0)
CRAN (R 4.0.4)
CRAN (R 4.0.3)
CRAN (R 4.0.4)
CRAN (R 4.0.4)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.2)
CRAN (R 4.0.3)
CRAN (R 4.0.2)
CRAN (R 4.0.0)
CRAN (R 4.0.3)
CRAN (R 4.0.4)
Github (34f43d0)
CRAN (R 4.0.3)
CRAN (R 4.0.0)
CRAN (R 4.0.2)
CRAN (R 4.0.3)
CRAN (R 4.0.0)
CRAN (R 4.0.4)
CRAN (R 4.0.0)
CRAN (R 4.0.0)

[1] /home/kevcaz/Applications/R_4_0
[2] /usr/local/lib/R/site-library
[3] /usr/lib/R/site-library
[4] /usr/lib/R/library

P ── Loaded and on-disk path mismatch.

</details>
@KevCaz
Copy link
Member Author

KevCaz commented Apr 29, 2021

BTW, on my fork all jobs are currently failing https://github.com/KevCaz/weathercan/actions/runs/796508465.

@KevCaz
Copy link
Member Author

KevCaz commented Apr 29, 2021

Looks like this is simply due to the fact that the database is currently unavailable! 😄
https://climate.weather.gc.ca/error/dbdown_e.html
Sorry, I should have checked that first!
I'll wait till it gets back and then I'll close this!

@KevCaz
Copy link
Member Author

KevCaz commented Apr 29, 2021

This makes me think that it could useful to add a simple function to check if data are available, something like:

  check_status <- function() {
    tmp <- httr::GET("https://climate.weather.gc.ca")
    chk <- tmp$url == "https://climate.weather.gc.ca/error/dbdown_e.html"
    if(chk) {
      message("service is down!")
    } else {
      message("data are available.")
    }
    invisible(!chk)
  }

just a thought.

@steffilazerte
Copy link
Member

That's a beautiful idea! I wasn't aware of the dbdown site, which makes it nice and easy to check! Would you be interested in submitting a pull request (matching the normals branch, since that's the one I'm just about to merge in for an upcoming update).

I'd be very grateful!

@KevCaz
Copy link
Member Author

KevCaz commented Apr 29, 2021 •
edited
Loading

Sure 😃 . I'll think a bit about where this feature would be the most useful in the package, and then I'll open a PR, soon (tomorrow or this week-end!).

@steffilazerte
Copy link
Member

Awesome! The packages is a bit messy, but in the normals branch (important to work on that one!), weather_html() (weather.R), normals_html() (normals.R), stations_dl_internal(), and normals_stn_list() (stations.R) are the four main access points for the ECCC. They all use the urls stored as options loaded by the .onLoad() in weathercan-pkg.R

Even if you just want to get it started, I'm happy to finish it up if you're busy. I appreciate anything you have time for :)

@KevCaz KevCaz mentioned this issue May 4, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@KevCaz @steffilazerte