[REVIEW]: covid19.analytics: An R Package to Obtain, Analyze and Visualize Data from the 2019 Corona Virus Disease Pandemic

[whedon]

Submitting author: @mponce0 (Marcelo Ponce)
Repository: https://github.com/mponce0/covid19.analytics/
Version: 2.1.0
Editor: @fboehm
Reviewer: @LogIN-, @JDRomano2
Archive: 10.5281/zenodo.4640307

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/43bab096ad574f4510a7258c20a1153d"><img src="https://joss.theoj.org/papers/43bab096ad574f4510a7258c20a1153d/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/43bab096ad574f4510a7258c20a1153d/status.svg)](https://joss.theoj.org/papers/43bab096ad574f4510a7258c20a1153d)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@LogIN- & @JDRomano2, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @fboehm know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @LogIN-

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@mponce0) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @JDRomano2

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@mponce0) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @LogIN-, @JDRomano2 it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

PDF failed to compile for issue #2995 with the following error:

Can't find any papers to compile :-(

[mponce0]

@whedon generate pdf from branch literature

[whedon]

Attempting PDF compilation from custom branch literature. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[fboehm]

@JDRomano2 and @LogIN- - I just wanted to check on the status of your reviews. Please let me know if I can assist with anything. Thanks again!

[JDRomano2]

Thanks very much @fboehm for the invitation to review this submission! Please see below for individual comments on each of the checklist items that I feel need attention:

General checks

Contribution and authorship: Has the submitting author (@mponce0) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

The contributions made by the second author are not clear, as they are not a contributor to the GitHub source repository, and their contributions are not described elsewhere in the paper. I would strongly recommend including information about author contributions in an "About" heading within the GitHub README, as well as current contact information for at least the primary author.

Documentation

Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?

Documentation is provided primarily by means of examples. No API documentation is given.

If a reference manual with API documentation is available on CRAN, there should be a link to it from the GitHub README and/or the project website.

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

I cannot find any community guidelines in the paper, the code repository, or the project's website.

The process for reporting bugs and issues is especially important to include, and should be clearly defined in the README. Ideally this would make use of GitHub's issue tracking functionality.

Software paper

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?

The summary is somewhat vague and nonspecific.

"With the emergence of a new pandemic worldwide, a novel strategy to approach it has also emerged."

What does 'approach it' mean? The paper should clearly describe the types of data-related problems that the software is meant to solve.

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?

The authors do describe the motivation behind the project in the context of other open-source COVID-19 data resources. However, the "Statement of Need" section needs to be cleaned up. Much of the text in here should go either into the Summary (keeping in mind the recommended edits mentioned above) or into a separate section following the Statement of Need describing the software's functionality.

The GitHub README file contains a fairly good description of the needs this software is meant to address, and some of that text can probably be reused in the paper.

Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?

The paper is in need of stylistic and typographical revisions. A few examples (not comprehensive) include:

• "Coronavirus"; not "Corona virus"
• "COVID-19"; not "CoViD19"
• "GitHub"; not "GitHUB"
• Capitalized letters at the beginning of each item in the bulleted lists
• "the R language"; not "the R Language and Environment for Statistical Computing" (which is just the title of the publication, not the actual language's title).
• Why is the word "backup" in bold on the first page?

These are just a few examples that stand out.

References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Some of the citations are incomplete. For example, the citation for "The Comprehensive R Archive Network" should include the journal name, author, publication date, volume, and issue number. These are easily found by clicking on the DOI link provided in the reference. Similar omissions are found in several of the other references.

I'm not sure that enough of the citations point to appropriate publications or data sources. For example, you list the Visual Capitalist infographic as one of the data sources used by your software, but the linked article doesn't seem to provide any way to access the data.

[mponce0]

Many thanks @JDRomano2 for the thorough review and useful comments!
The editions and updates addressing this reviewer's comments have been added to the manuscript on the "literature" branch of the https://github.com/mponce0/covid19.analytics/ repository.

General checks

Contribution and authorship: Has the submitting author (@mponce0) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

The contributions made by the second author are not clear, as they are not a contributor to the GitHub source repository, and their contributions are not described elsewhere in the paper. I would strongly recommend including information about author contributions in an "About" heading within the GitHub README, as well as current contact information for at least the primary author.

Authors contributions were (and are) originally described in the "Description" file, which is the standard place for doing it in R packages.

Following the reviewer's recommendation we have also added an "About" heading to the README document stating more clearly the contributions of the authors, as well as the way for the community to contribute to the package.

Documentation

Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?

Documentation is provided primarily by means of examples. No API documentation is given.

If a reference manual with API documentation is available on CRAN, there should be a link to it from the GitHub README and/or the project website.

A link has been added to the README for API documentation, in this case, functions descriptions from CRAN immediately after the Overview of the Main Functions from the "covid19.analytics" Package section.

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

I cannot find any community guidelines in the paper, the code repository, or the project's website.

The process for reporting bugs and issues is especially important to include, and should be clearly defined in the README. Ideally this would make use of GitHub's issue tracking functionality.

This was mentioned both in the paper as well as in the README file, in the first bullet point of "Statement of need":
Users can also submit tickets for bugs, suggestions or comments using the "issues" tab" --referring to the GitHub repository--.
We have also added a paragraph to the paper about how to seek for support using the "ISSUES tracker" and contribute via "pull request".
As a side note please notice that both features have been in place and used, as you can verify in the repo (https://github.com/mponce0/covid19.analytics/issues) since quite some time ago.

Software paper

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?

The summary is somewhat vague and nonspecific.

"With the emergence of a new pandemic worldwide, a novel strategy to approach it has also emerged."

What does 'approach it' mean? The paper should clearly describe the types of data-related problems that the software is meant to solve.

We have clarified the summary and added a few sentences to describe what the package can allow users to achieve in terms of the data availability and its challenges.

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?

The authors do describe the motivation behind the project in the context of other open-source COVID-19 data resources. However, the "Statement of Need" section needs to be cleaned up. Much of the text in here should go either into the Summary (keeping in mind the recommended edits mentioned above) or into a separate section following the Statement of Need describing the software's functionality.

The GitHub README file contains a fairly good description of the needs this software is meant to address, and some of that text can probably be reused in the paper.

We have restructured and edited the manuscript to incorporate the reviewer's comments.

Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?

The paper is in need of stylistic and typographical revisions. A few examples (not comprehensive) include:

• "Coronavirus"; not "Corona virus"
• "COVID-19"; not "CoViD19"
• "GitHub"; not "GitHUB"
• Capitalized letters at the beginning of each item in the bulleted lists
• "the R language"; not "the R Language and Environment for Statistical Computing" (which is just the title of the publication, not the actual language's title).
• Why is the word "backup" in bold on the first page?

These are just a few examples that stand out.

We thank the reviewer for noticing this, and we have revisit and edited the manuscript to fix all of these.

References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Some of the citations are incomplete. For example, the citation for "The Comprehensive R Archive Network" should include the journal name, author, publication date, volume, and issue number. These are easily found by clicking on the DOI link provided in the reference. Similar omissions are found in several of the other references.

We have reviewed the citations and completed the information for the CRAN publication. We couldn't identify any other reference missing any of the elements described above.

I'm not sure that enough of the citations point to appropriate publications or data sources. For example, you list the Visual Capitalist infographic as one of the data sources used by your software, but the linked article doesn't seem to provide any way to access the data.

This is because the data from the infographics was gather manually, ie. no actual data repository is offered.
WRT the other data sources, we will be OK to follow any other recommendation in how to better reference/cite these data sources.

[mponce0]

@whedon generate pdf from branch literature

[whedon]

Attempting PDF compilation from custom branch literature. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[mponce0]

@whedon generate pdf from branch literature

[whedon]

Attempting PDF compilation from custom branch literature. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[whedon]

👋 @JDRomano2, please update us on how your review is going.

[whedon]

👋 @LogIN-, please update us on how your review is going.

[LogIN-]

Hi @fboehm thank you for invitation for the review.

Authors did a good job in developing a package, it can be successfully installed and used.

Here is my opinion on what needs to be changed:

• Authors should remove self-citation to the same article available to preprint server (Ponce, 2020)

When this self-citation is removed authors should describe in more detail functionality of this R package, similar to their preprint publication, otherwise from current paper many aspects of the package will stay unclear.

---

Maybe the best solution is to have peer review of their preprint paper and having that published in JOSS.

[fboehm]

@LogIN- thanks for your message. Can you clarify - is the preprint article the same (same title, same text) that is being considered here? Or does it merely have the same title? Thanks again!

[mponce0]

@LogIN- thanks for your message. Can you clarify - is the preprint article the same (same title, same text) that is being considered here? Or does it merely have the same title? Thanks again!

The title is the same but the pre-print article (~47 pages) is quite lengthy. When considering submitting the manuscript to JOSS we thought about it but when checking the structure and recommended format we felt that it wouldn't fit with this format.
Here is a link to the pre-print https://arxiv.org/abs/2009.01091
We will happy to submit the arXiv version instead if that's OK?

[mponce0]

Hi @fboehm thank you for invitation for the review.

Authors did a good job in developing a package, it can be successfully installed and used.

Here is my opinion on what needs to be changed:

• Authors should remove self-citation to the same article available to preprint server (Ponce, 2020)

When this self-citation is removed authors should describe in more detail functionality of this R package, similar to their preprint publication, otherwise from current paper many aspects of the package will stay unclear.

Maybe the best solution is to have peer review of their preprint paper and having that published in JOSS.

Hi @LogIN- , many thanks for your review and encouraging words.
We just wanted to clarify and expand a little bit on our previous comment to @fboehm , we did not submit the original manuscript from arXiv to JOSS as we thought the format and in particular the length of it wouldn't fit the style of JOSS.
In addition to that, we thought of using the JOSS publication as the main point of references to the "code/package" itself while working in additional publications more aimed to applications and introductory tutorials in how to use the package itself. In particular, considering that we will continue working and adding new features to the package we thought of having a very high level description and reference to the code in this JOSS publication.
As mentioned above, we are OK with submitting the longer manuscript as show in the arXiv version, but even considering previous JOSS publications (eg. https://doi.org/10.21105/joss.02376) we decided to submit this current shorter version of the paper.

[fboehm]

Yes, that's right, @mponce0 . The purpose of JOSS is to publish the software and, typically, we prefer short articles that discuss the software.

@LogIN- I think it would be ok, and, possibly helpful, to maintain the citation to the preprint, especially considering that it's not the same as the manuscript that accompanies the software submission to JOSS. I agree, @LogIN-, with you encouraging the authors to detail functionality of the R package, while aiming to keep the JOSS manuscript length at ~ 3 pages or less.

Does that seem reasonable?

[mponce0]

Yes, that's right, @mponce0 . The purpose of JOSS is to publish the software and, typically, we prefer short articles that discuss the software.

@LogIN- I think it would be ok, and, possibly helpful, to maintain the citation to the preprint, especially considering that it's not the same as the manuscript that accompanies the software submission to JOSS. I agree, @LogIN-, with you encouraging the authors to detail functionality of the R package, while aiming to keep the JOSS manuscript length at ~ 3 pages or less.

Does that seem reasonable?

I'm certainly more inclined to keep the JOSS paper short, we are already at 5 pages (a bit above 3 without including references).
We will change the title of the pre-print to reflect that arXiv article is the "User's Manual and Examples of Applications" and the JOSS paper as the main reference of the package.

@fboehm There are two main issues with adding the description for the functions:

1. it will increase the length of the manuscript (we added some parts due to some at the request from other reviewer),
2. my main concern is that we will keep adding new functions and capabilities in the future as the situation unfolds and as requested by users (we have already done this a couple of times) and we develop further projects related to the package. Then the description of the functions presented in this paper would turn to be incomplete.

Personally, I think the best would be to use the JOSS paper as the main reference to the package without going into the details of the implementation of the API and allowing users to get the explicit details from the package website and documentation.
As mentioned before, I know other paper at JOSS follow this very same approach.

[mponce0]

@fboehm WRT this last point about the functionality of the package, we went back to the manuscript and try to add a few more sentences detailing some of the functionalities at a very "high" level (end of the "Functionalities and Main Features" section).

Please let us know what do you think.

[mponce0]

@whedon generate pdf from branch literature

[whedon]

Attempting PDF compilation from custom branch literature. Reticulating splines etc...

[danielskatz]

@whedon accept from branch literature

[whedon]

Attempting dry run of processing paper acceptance...

[mponce0]

I've also created mponce0/covid19.analytics#10 for a bunch of grammar/style issue - let me know what you disagree with, or merge this.

Merged! Many thanks for the thorough review and suggestions/edits.

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1002/wics.1212 is OK
- 10.1093/nar/gkx1095 is OK
- 10.1038/d41586-020-01695-w is OK
- 10.1038/s41597-020-0524-5 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#2178

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2178, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true from branch literature 

[danielskatz]

Re ihaka:1996, you didn't change the tag to doi - it's still url

[mponce0]

Re ihaka:1996, you didn't change the tag to doi - it's still url

Strange, this was done in this commit
mponce0/covid19.analytics@847d626

and the actual version in the repo, https://github.com/mponce0/covid19.analytics/blob/literature/JOSS/references.bib shows the change...
Maybe it got lost from the merge?

[danielskatz]

I see the https://doi.org part was removed, but the tag is url, not doi

[mponce0]

So sorry, I missed that.. should be fixed now.

[danielskatz]

@whedon accept from branch literature

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1080/10618600.1996.10474713 is OK
- 10.1002/wics.1212 is OK
- 10.1093/nar/gkx1095 is OK
- 10.1038/d41586-020-01695-w is OK
- 10.1038/s41597-020-0524-5 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#2179

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2179, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true from branch literature 

[danielskatz]

@whedon accept deposit=true from branch literature

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.02995 joss-papers#2180
1. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02995
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[danielskatz]

Congratulations to @mponce0 (Marcelo Ponce) and co-author!

And thanks to @LogIN- and @JDRomano2 for reviewing, and @fboehm for editing!

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02995/status.svg)](https://doi.org/10.21105/joss.02995)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02995">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02995/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02995/status.svg
   :target: https://doi.org/10.21105/joss.02995

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[mponce0]

Congratulations to @mponce0 (Marcelo Ponce) and co-author!

And thanks to @LogIN- and @JDRomano2 for reviewing, and @fboehm for editing!

Thank you very much to you all: @JDRomano2 @danielskatz @LogIN- @fboehm