Unable to replicate results in openjournals/joss-reviews#6366 article proof

[ifoxfoot]

openjournals/joss-reviews/issues/6366

Hello. I am able to install the fqar package both via github and CRAN. I believe I am using the package downloaded from CRAN (they both have the same version number so I can't tell).

I am able to run the code chicago_invs <- assessment_list_inventory(chicago_fqas) which returns a large list with 309 elements.

However, when I run the next line of code chicago_cooccurrence <- assessment_cooccurrences(chicago_invs) I get a dataframe with 0 observations and 9 variables.

All of the rest of the code in the article proof gives errors.

My internet is not the greatest and it did cut out twice during the download of the Chicago fqas. Also my R version is a little behind. See session info below. Let me know if that could be the issue? Thanks!

> sessionInfo()
R version 4.3.0 (2023-04-21 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default


locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] fqar_0.5.1

loaded via a namespace (and not attached):
 [1] vctrs_0.6.2      httr_1.4.7       cli_3.6.1        rlang_1.1.1      purrr_1.0.1     
 [6] generics_0.1.3   jsonlite_1.8.8   glue_1.6.2       colorspace_2.1-0 scales_1.3.0    
[11] fansi_1.0.4      grid_4.3.0       munsell_0.5.0    tibble_3.2.1     fastmap_1.1.1   
[16] lifecycle_1.0.4  memoise_2.0.1    compiler_4.3.0   dplyr_1.1.2      pkgconfig_2.0.3 
[21] tidyr_1.3.1      rstudioapi_0.14  R6_2.5.1         tidyselect_1.2.0 utf8_1.2.3      
[26] pillar_1.9.0     magrittr_2.0.3   tools_4.3.0      gtable_0.3.4     ggplot2_3.4.2   
[31] cachem_1.0.8

[equitable-equations]

@ifoxfoot thanks for reaching out, as well as for your helpful notes on the paper itself, which I'll reply to as soon as possible. I was just able to run those commands successfully on my Macbook, with chicago_cooccurrence <- assessment_cooccurrences(chicago_invs) giving a data frame with 4,463,025 rows.

If the later commands (species_profile and species_profile_plot) are giving errors as well, then something has likely gone wrong at the downloading step. We've worked hard to make our functions robust against interruptions, but it's totally possible something has slipped. I'll do some more digging.

Again, thank you for your work!

[ifoxfoot]

Hi @equitable-equations I'm also testing it on my work computer which has some restrictions--I don't have admin status. I can try it on my personal computer with a more updated R version soon. That could also be the issue. Thank you for your prompt reply!

[equitable-equations]

Another thing is that we intentionally chose a large and popular FQA database when writing this paper, which has the side effect of making the code slower and the outputs very large. In other words, we weren't really thinking about readers copy/pasting our code. We could consider using a smaller example which would run more reliably on older systems.

[ifoxfoot]

Update: I mistyped in my orignal issue. I get a large list from the line chicago_fqas <- download_assessment_list. I was getting an empty data frame from the line chicago_invs <- assessment_list_inventory(chicago_fqas). Sorry about that!

I ran the code with no internet interruptions and it worked on the same system so that is good.

Then I ran it again and interrupted the connection by disconnecting and connecting to the internet. The console looks like this

> chicago_fqas <- download_assessment_list(database_id = 80)
  |=====                                                                                                                                                                                                                                                                                                                |   2%Unable to connect. Please check internet connection.
  |=====================================================================================================================================================================================================================================================================================================================| 100%

Again I was able to download a large list with 309 elements. It looks like a successful download but it must not be.

then when I run the next line of code I get

chicago_invs <- assessment_list_inventory(chicago_fqas)
assessment_list must be a list of dataframes obtained from universalFQA.org. Type ?download_assessment_list for help.

Also chicago_invs is a df with 0 observations and 9 variables.

So I think the issue is that when the download function has a lapse in internet connection it cannot recover. It's not a huge issue so I will continue with the review. Thanks!

[equitable-equations]

Ok, great. I'm adding this to my to-do list for the next update. Thanks!

[mhesselbarth]

FYI it worked like a charm on my side 👍

[equitable-equations]

I've re-tested everything carefully and things appear to be ok. I'm closing this issue.