[REVIEW]: Leafmap: A Python package for interactive mapping and geospatial analysis with minimal coding in a Jupyter environment

[whedon]

Submitting author: @giswqs (Qiusheng Wu)
Repository: https://github.com/giswqs/leafmap
Version: v0.3.5
Editor: @ajstewartlang
Reviewers: @martinfleis, @TomasBeuzen
Archive: 10.5281/zenodo.5137372

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/b081afbd9bdaa4386475c787db8e370c"><img src="https://joss.theoj.org/papers/b081afbd9bdaa4386475c787db8e370c/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/b081afbd9bdaa4386475c787db8e370c/status.svg)](https://joss.theoj.org/papers/b081afbd9bdaa4386475c787db8e370c)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@TomasBeuzen & @martinfleis, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @ajstewartlang know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @TomasBeuzen

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@giswqs) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @martinfleis

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@giswqs) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @TomasBeuzen, @TomasBeuzen, @martinfleis it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.rse.2017.06.031 is OK
- 10.21105/joss.02272 is OK
- 10.5334/jors.148 is OK
- 10.5281/zenodo.4569086 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.47 s (174.0 files/s, 143442.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JSON                             6              0              0          50166
Python                          15           1573           1782           6213
XML                              2              0              0           1636
Markdown                        23            253              0            676
Jupyter Notebook                24              0           4369            479
YAML                             9             25             10            278
HTML                             2              3              0             65
TeX                              1              4              0             59
-------------------------------------------------------------------------------
SUM:                            82           1858           6161          59572
-------------------------------------------------------------------------------


Statistical information for the repository '4dcb6ecfa0ee11557c92d6e1' was
gathered on 2021/06/25.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Kharude, Sachin                  7          1145            131           11.60
Nahid Pervez                     3            32              4            0.33
Qiusheng Wu                     45          9258            433           88.08

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Kharude, Sachin             707           61.7          0.4                3.54
Nahid Pervez                 29           90.6          0.4                6.90
Qiusheng Wu                8832           95.4          0.8                6.13

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[giswqs]

@ajstewartlang I noticed TomasBeuzen is listed twice under the Reviewers section in the PDF. Is it possible to correct this?

[danielskatz]

👋 @ajstewartlang - FYI, this is because in #3396 you both assigned him and then added him - you only need to do one of these: typically, use assign for the first reviewer, and then add for any subsequent ones. You can probably use @whedon remove @TomasBeuzen as reviewer to remove one of the duplicates here, then regenerate the PDF. Let me know if you have problems...

[whedon]

👋 @martinfleis, please update us on how your review is going (this is an automated reminder).

[whedon]

👋 @TomasBeuzen, please update us on how your review is going (this is an automated reminder).

[whedon]

👋 @TomasBeuzen, please update us on how your review is going (this is an automated reminder).

[martinfleis]

Dear @giswqs and @ajstewartlang,

I have finished the review of the package and its paper. Even though the package is an advanced piece of software helpful on many occasions, some issues need to be resolved before publication in JOSS. Please find a detailed explanation below.

Review

leafmap will undoubtedly find its audience. It does what it promises to do and does it well. I have only a few remarks, mostly minor, but let me start with the only major issue.

The software, in its current state, can be considered untested. The repository contains a few test files, but those do not sufficiently check the functionality. As reported by pytest-cov, overall coverage is only 12% (see the full report below). I would argue that even those 12% do not reflect the actual state as most of the tests are essentially checking if it fails or not. Test suite needs to check whether the output of each class, method and function returns what is expected of it. The current test suite does not do that. I understand that it may not be as straightforward in interactive mapping, but there are ways based on string (HTML) outputs - look at folium, for example.

Name                   Stmts   Miss  Cover
------------------------------------------
leafmap/__init__.py       16      3    81%
leafmap/basemaps.py       28      4    86%
leafmap/colormaps.py      74     74     0%
leafmap/common.py       1038    839    19%
leafmap/foliumap.py      470    470     0%
leafmap/heremap.py       153    153     0%
leafmap/leafmap.py       932    809    13%
leafmap/legends.py        27     24    11%
leafmap/osm.py            88     50    43%
leafmap/toolbar.py       549    549     0%
------------------------------------------
TOTAL                   3375   2975    12%

Now on to minor issues (in no particular order):

• XYZ tiles require attribution. Without requiring one, leafmap may violate terms of use.
• I would welcome some explanation of different backends in the introduction, not only in the FAQ. In the Get Started notebook, there is no explanation, leaving the newcomer confused. It should also be mentioned that backends do not offer equal functionality.
• Methods supported by one backend and not the other should raise an informative warning, as NotImplementedError with troubleshooting (use the other backend), instead of an AttributeError.

AttributeError: module 'leafmap.foliumap' has no attribute 'linked_maps'`

• Most of the notebooks in the Tutorials section of the documentation would benefit from some descriptive text accompanying the code. Unfortunately, many of them do not contain any, and not all of them are self-explanatory.
• Just for a reference, one of my earlier issues has already been resolved ().

The mental model leafmap use is very different from the PyData stack I am mostly used to. It tries to solve everything, which is even taken into extremes with functions like csv_to_pandas which is just pd.read_csv(in_csv, **kwargs) or read_postgis which is a wrapper of gpd.read_postgis(sql, con, geom_col, crs, **kwargs). It is not a problem that they exist; I am just struggling to figure out whether they should be a part of the public API as they are now or whether they are considered private. The same applies to a series of functions I would not expect in the API of a package like leafmap, supporting the dev-level installation of packages (update_package, install_from_github...). Again, should it be part of a user-facing API? I am not asking for these to be changed; I just wanted to point out that it is not a very common practice.

Paper

A few notes regarding the paper:

• The usage of bullet points with hyperlinks and without much description is a rather strange choice. I would recommend reformulating those sections. The paper needs to be readable without the necessity to click on any link in there.
• The package itself is only briefly explained. A paragraph with a more detailed description outlining different modules, backends and internal logic of the library would help.
• folium should be cited
• ipyleaflet should be cited
• ipywidgets should be cited

[giswqs]

@martinfleis Thank you very much for the thorough review and constructive comments. I will revise the repo and paper as suggested over the next few days.

Regarding your major comment on test coverage, I will look into the folium package to see how to increase the test coverage in interactive mapping. As you already pointed out, test coverage in interactive mapping is not very straightforward. I followed the ipyleaflet package for testing coverage in interactive mapping. The ipyleaflet repo itself does not contain the tests folder. All the tests are actually performed when the documentation website is being built and deployed to https://ipyleaflet.readthedocs.io, which is similar to what I did for leafmap. All the notebook examples demonstrating key functions of leamap are executed and tested using GitHub workflow (docs.yml) when deployed to GitHub Pages, e.g., https://leafmap.org/notebooks/01_leafmap_intro. Users can see the interactive map. The downside is that pytest coverage can't capture these automated tests passed by GitHub workflow.

Similarly, the here-map-widget-for-jupyter package repo does not contain a tests folder. All the tests are performed when the documentation website is being built and deployed to https://here-map-widget-for-jupyter.readthedocs.io

[ajstewartlang]

👋 @ajstewartlang - FYI, this is because in #3396 you both assigned him and then added him - you only need to do one of these: typically, use assign for the first reviewer, and then add for any subsequent ones. You can probably use @whedon remove @TomasBeuzen as reviewer to remove one of the duplicates here, then regenerate the PDF. Let me know if you have problems...

Thanks @danielskatz - yes, looks like I used assign twice rather than add the second time - I will remove the duplicate and regenerate the PDF....

[ajstewartlang]

@whedon remove @TomasBeuzen as reviewer

[whedon]

OK, @TomasBeuzen is no longer a reviewer

[ajstewartlang]

@whedon generate pdf

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[ajstewartlang]

@whedon generate pdf

[ajstewartlang]

@whedon check references

[giswqs]

@whedon generate pdf

[ajstewartlang]

@whedon generate pdf

@giswqs I've reached out to the rest of the JOSS team to find out why whedon isn't generating the pdf...

[giswqs]

@ajstewartlang Thank you for fixing the typo. I have made a tag version that includes all changes made to the paper.

• Tagged version: v0.3.5
• Zenodo link: https://doi.org/10.5281/zenodo.5137372
• DOI: 10.5281/zenodo.5137372

[danielskatz]

@whedon check references

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.rse.2017.06.031 is OK
- 10.21105/joss.02272 is OK
- 10.5334/jors.148 is OK
- 10.5281/zenodo.4569086 is OK
- 10.5281/zenodo.4447642 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.rse.2017.06.031 is OK
- 10.21105/joss.02272 is OK
- 10.5334/jors.148 is OK
- 10.5281/zenodo.4569086 is OK
- 10.5281/zenodo.4447642 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[ajstewartlang]

@whedon set 10.5281/zenodo.5137372 as archive

[whedon]

OK. 10.5281/zenodo.5137372 is the archive.

[ajstewartlang]

Many thanks @martinfleis and @TomasBeuzen for your helpful and detailed reviews - thank you @giswqs for addressing all the comments - I've read through myself and agree that this is a great submission that will be very useful to many people! Congrats and thank you for submitting to JOSS!

[ajstewartlang]

@whedon set v0.3.5 as version

[whedon]

OK. v0.3.5 is the version.

[ajstewartlang]

@whedon recommend-accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.rse.2017.06.031 is OK
- 10.21105/joss.02272 is OK
- 10.5334/jors.148 is OK
- 10.5281/zenodo.4569086 is OK
- 10.5281/zenodo.4447642 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#2482

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2482, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[kyleniemeyer]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.03414 joss-papers#2484
1. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.03414
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[kyleniemeyer]

Congratulations @giswqs on your article's publication in JOSS!

Many thanks to @martinfleis and @TomasBeuzen for reviewing this submission, and @ajstewartlang for editing.

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.03414/status.svg)](https://doi.org/10.21105/joss.03414)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.03414">
  <img src="https://joss.theoj.org/papers/10.21105/joss.03414/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.03414/status.svg
   :target: https://doi.org/10.21105/joss.03414

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss