[REVIEW]: EmissV: An R package to create vehicular and other emissions by Top-down methods to air quality models

[whedon]

Submitting author: @Schuch666 (Daniel Schuch)
Repository: https://github.com/atmoschem/EmissV
Version: 0.664.3
Editor: @arfon
Reviewer: @nuest, @benmarwick
Archive: 10.5281/zenodo.1451027

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/071d027997ac93d8992099cb5010a044"><img src="http://joss.theoj.org/papers/071d027997ac93d8992099cb5010a044/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/071d027997ac93d8992099cb5010a044/status.svg)](http://joss.theoj.org/papers/071d027997ac93d8992099cb5010a044)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@nuest, @benmarwick, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @arfon know.

Review checklist for @nuest

Conflict of interest

• As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Version: Does the release version given match the GitHub release (0.664.3)?
• Authorship: Has the submitting author (@Schuch666) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @benmarwick

Conflict of interest

• As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Version: Does the release version given match the GitHub release (0.664.3)?
• Authorship: Has the submitting author (@Schuch666) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon. I'm here to help you with some common editorial tasks. @nuest it looks like you're currently assigned as the reviewer for this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

--> Check article proof 📄 <--

[nuest]

Sorry, I'm at a conference this week. Will complete this next week!

[arfon]

@nuest @benmarwick - friendly reminder to complete your reviews when you get a chance 😁

[benmarwick]

Thanks, working on it!

[Schuch666]

Hey guys!

Do not forget me

This is a kind reminder
:)

[arfon]

@nuest @benmarwick - friendly reminder to complete your reviews when you get a chance 😁

Yes, please take a look at this when you get a chance please!

[nuest]

Sincere apologies for the delay, too many GH emails and this one fell through the cracks. I started my review and have some first comments that can be addressed rightaway, see issues linked above.
I'll start the functional evaluation next.

[nuest]

I gave the paper.md a first read, some comments/questions:

• Why are the emission inventories the scapegoat? Why not the athmospheric data?
• Have you tested the package with data other than for Sao Paulo State?
• I recommend not to use abbreviations in the example code for object names ("SP", "MG"), makes it harder to follow.
• The file sample.tiff is missing, probably must be dmsp.tiff
• The logging output from areaSource differs on my machine compared to what the paper has, starting at the fourth decimal place... is this relevant or expected?
• The scale of the created plot differs for Fig. 1:

• The code to create the second figure is missing.
• The final command fails on my machine:

> CO_emissions <- emission(TOTAL,"CO",list(SP = SP, MG = MG),grid,mm=28, plot = T)
[1] "calculating emissions for CO using molar mass = 28 ..."
Error in seq.default(zrng[1], zrng[2], length.out = cuts + 2) : 
  'from' must be a finite number
In addition: Warning messages:
1: In asMethod(object) :
  complete map seems to be NA's -- no selection was made
2: In min(x) : no non-missing arguments to min; returning Inf
3: In max(x) : no non-missing arguments to max; returning -Inf

• References
  • Eixport has a JOSS paper with a DOI, which should be used for the citation
  • Pulles & Heslinga also has a DOI: doi.org/10.13140/RG.2.1.2082.8007
  • Why do you cite ncdf4, but not other packages (raster, sp?)

In my opinion the submission needs some work. I am also not a domain expert, so I cannot say anything about the validity of the method included in the package.

@benmarwick Are you familiar with the methodology and kind of analysis the package performs?

[benmarwick]

Yes, to start with, I'd love to see another attempt at a clear statement of what problems the software is designed to solve and who the target audience is? Who are the users likely to be? I've just done a tiny bit work helping students in atmospheric chemistry, so the methods here are not familiar to me.

I can run the example code until here:

TOTAL  <- totalEmission(veiculos,EF,pol = c("CO"),verbose = T)
Error: $ operator is invalid for atomic vectors 
5.
as.character.units(structure(1127549.32468524, units = structure(list(
    numerator = "t", denominator = "year"), .Names = c("numerator", 
"denominator"), class = "symbolic_units"), class = "units")) 
4.
as.character(structure(1127549.32468524, units = structure(list(
    numerator = "t", denominator = "year"), .Names = c("numerator", 
"denominator"), class = "symbolic_units"), class = "units")) 
3.
paste("Total of", pol[i], ":", sum(total_t_y), units::deparse_unit(total_t_y)) 
2.
print(paste("Total of", pol[i], ":", sum(total_t_y), units::deparse_unit(total_t_y))) at totalEmission.R#62
1.
totalEmission(veiculos, EF, pol = c("CO"), verbose = T)

I've noted this at the repo here: atmoschem/EmissV#16

Update: seems I was missing some Linux libraries. I'll continue testing on Windows.

[Schuch666]

Thank @nuest and @benmarwick

I'm working on the text (and the code). Soon I will update the package and write here some detailed explanations

[Schuch666]

Hi @nuest and @benmarwick, thanks for the suggestions, some corrections have already been made to the package and the suggestions in the article have already been incorporated.

@nuest I make some changes in the paper (and in the repository)

• added to the paper a reason to emission inventories being the scapegoat of air quality model input
• The package was tested for some cases in São Paulo and some Brazil adjacent states, for now, the main reason is that São Paulo has a Pollution problem and an available air quality monitory network to compare the results
• I changed the abbreviation for states names
• that's right, the file sample.tiff was changed for dmsp.tiff, this is a smaller file (for CRAN submission).
• The differences of area source are mainly because of the difference of resolution of the image and the loss of accuracy, it is not a problem (but is an updated version).
• The figure 1 shows g/d on each image pixel.
• The second figure is created in the emission function with plot = T, the reason is that output is a matrix with can be written in model input files (with the cited packages) and is not georeferenced
• About the error: the function rasterSource was updated (to always deal with NAs)
• The References was added to paper, I cited the packages that can be written in netcdf format files, eixport has specific functions for write in wrf input files

@benmarwick the main problem is the need of input data for air quality models, depending on the region there is no information about the temporal or spatial distribution of the vehicular emissions or in some cases, the information is at a national level and not updated for decades. The audience is students and researchers who are interesting in using numerical air quality models but don't have a good inventory or wants an alternative inventory.

The function vehicles have more detailed description and the number of vehicles is integer now.

I'm Working on the package remain Issues!

[Schuch666]

Hi @nuest , @benmarwick and @arfon

I have some news: EmissV was accepted on CRAN.

The paper was revised:

• added more detailed description of the problem and the package
• the example was slightly reformulated (the dmsp_hi-res.tiff was added to the repository and example)
• added references
• grammatical and orthographic corrections

When you have time, take a look.

thank you

PS: We are planning the use of EmissV in some Argentine cities in the next months.

[arfon]

I have some news: EmissV was accepted on CRAN.

Great! Thanks for the update @Schuch666.

@nuest, @benmarwick - could you come and take another look at this sometime in the next couple of weeks?

[Schuch666]

news:

Automated tests added to the package

[arfon]

Ping @nuest, @benmarwick 😄 please come any take another look at this package sometime soon.

[Schuch666]

Hey guys @nuest @benmarwick

Please come any take another look at this package

This is a kindly reminder

[ibarraespinosa]

hi @arfon, I'm co-author of this paper and package. Is it possible to speed up the review process? Thanks! Sergio

[arfon]

Friendly reminder to come back to this review sometime soon please @nuest & @benmarwick

[nuest]

@Schuch666 Thanks for taking the time to work on our suggestions, and for your understanding that this takes longer than it should. Congrats for getting the package on CRAN!

@whedon generate pdf

[nuest]

Ok, so PDF re-generation does not seem to work... @arfon did I miss something here?

• Don't want to be too picky on this, but for me the reason to make emission the "scapegoat" must be connected to the other measured variables somehow. Is it the case that athmosphere is not blamed because these values are easier to measure? Is the landscape not blamed because it is not changing? This might be obvious for you, but IMO the JOSS abstract should be understandable for most people.
• I can now compile a PDF from paper.md without any issues - good!
• I get slightly different results in some outputs - is that an issue? See e.g.

> TOTAL  <- totalEmission(fleet,EF,pol = c("CO"),verbose = T)
Total of CO : 1128297.0993334 t year-1 

• regarding the differences in the outputs - would you consider updating the paper with a short statement (or the function docs) that the outputs might differ slightly because of... X
• I'd like to restate Ben's point about the statement of need. AFAICS the first paragraph is almost there. Do you want to help to better understand anthropogenic emissions? Do you want to stop the emissions being used as a scapegoat? What would a user want to... "estimates vehicular emissions by a top-down approach". This is probably totally clear to you, but the first paragraph can just as well welcome any audience.

@arfon Does the author have to make a new release on GitHub? Not sure how to answer the "matches" checklist item. Thanks!

[arfon]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

--> Check article proof 📄 <--

[arfon]

Ok, so PDF re-generation does not seem to work... @arfon did I miss something here?

☝️ try @whedon generate pdf

@arfon Does the author have to make a new release on GitHub? Not sure how to answer the "matches" checklist item. Thanks!

Basically this is a question about the version number you see here ( 0.664.3) and what you find on GitHub.

[arfon]

Hi @benmarwick - do you know when you will be able to complete your review by?

[nuest]

I see a few minor questions and some need for rephrasing, as described in #662 (comment), other than that I'm done.

[Schuch666]

Hey @nuest , thanks for the suggestions.

The reason itself of emission being the "scapegoat" is that a difficult and highly multidisciplinary task. In the case of the anthropogenic emissions, the main difficulty is to create an updated inventory (sometimes there is no inventory available) with the available information (sometimes there is a very low amount of information available). I think one reason can be that the responsibility for the emissions is not of the model user in many cases.

I added some text to the Introduction:

'The anthropogenic emissions, especially vehicular emissions, are highly dependent on human activity and constantly changing due to various factors ranging from economic (such as the state of conservation of the fleet, renewal of the fleet and the price of fuel) to legal aspects (such as the vehicle routing).'

Atmospheric models are continuously developed by research institutions. As well as ways to assimilate a wide variety of data (atmospheric, terrestrial, vegetation, etc.) as satellite images, classifications and parameterizations. The EmissV allow the user to create the emissions with data like tables, shapefiles and images this give the user some power to use available information and more responsibility.

A statement of need of the package use has added next to this:

'The EmissV is an R package that estimates vehicular emissions by a top-down approach, the emissions are calculated using the statistical description of the fleet at available level (National, State, City, etc). The following steps show an example of the workflow for calculating vehicular emissions, this emissions are initially temporally and spatially disaggregated, and then distributed spatially and temporally to be used as input in numeric air quality models such WRF-Chem [@Grelletal2005].'

The values slightly different results in some outputs had updated, the reason it can be that the function vehicles operate on integer numbers only now (that make more sense to me).

If any point can be improved let me know

PS: I changed the 'veiculos' for fleet

[Schuch666]

Hi, @benmarwick , are you still with us?

[nuest]

@Schuch666 Please double check the documentation to mention that output values might differ slightly. If users run your algorithms and compare outcomes (e.g. in a reproducible research set-up, a reviewer runs an article manuscript) they should be aware that small differences can occur and why.

That's all from me then: LGTM 👍

[arfon]

@Schuch666 Please double check the documentation to mention that output values might differ slightly. If users run your algorithms and compare outcomes (e.g. in a reproducible research set-up, a reviewer runs an article manuscript) they should be aware that small differences can occur and why.

Once you've addressed this comment @Schuch666 I think we're good to accept here.

[Schuch666]

thank you @arfon

I put a note in the text talking about these differences that can appear in the calculated values and in the figures due to changes between versions of the package (in the code or in the sample files) and also of the version of its dependencies.

[Schuch666]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

👉 Check article proof 📄 👈

[arfon]

@Schuch666 - At this point could you make an archive of the reviewed software in Zenodo/figshare/other service and update this thread with the DOI of the archive? I can then move forward with accepting the submission.

[Schuch666]

Hi @arfon

I upload to Zenodo with the following DOI:
10.5281/zenodo.1451027

best regards

[arfon]

@whedon set 10.5281/zenodo.1451027 as archive

[whedon]

OK. 10.5281/zenodo.1451027 is the archive.

[arfon]

@nuest - many thanks for your review here ✨

@Schuch666 - your paper is now accepted into JOSS and your DOI is https://doi.org/10.21105/joss.00662 ⚡ 🚀 💥

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](http://joss.theoj.org/papers/10.21105/joss.00662/status.svg)](https://doi.org/10.21105/joss.00662)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.00662">
  <img src="http://joss.theoj.org/papers/10.21105/joss.00662/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: http://joss.theoj.org/papers/10.21105/joss.00662/status.svg
   :target: https://doi.org/10.21105/joss.00662

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://www.flipcause.com/secure/cause_pdetails/Mjk3Nzk=