What we need for a first PyGMT paper?

[leouieda]

In #653 we started discussing a possible PyGMT paper. The key question is "when should we publish our first paper?". Here are a few things we need before that can be done:

• An authorship policy. It's really important that we agree on this before doing any work a paper. See the Fatiando policy which was generated after long discussion and research into similar policies. (Done at Add authorship policy #726)
• Define a minimum viable product. What does PyGMT need to have in order to be fully usable? I'd say we're pretty close to having a usable alternative to Cartopy (if not there already). We don't necessarily need a "complete" product (when is software ever "done"?) but the package should be in usable state.
• Documentation! There is a lot of work to be done here and if we ignore the docs now and keep piling on features it will be hard to do it later. I would suggest basing the structure on the Divio system. Thinking about this now will help us define the mvp.
• Decide on a journal. I see 2 options:
  • JOSS (pro = easy + free + actual code review | con = not as well known yet)
  • G³ (pro = highly visible in geoscience + GMT is already published there | con = expensive + no code review)

[weiji14]

The key question is "when should we publish our first paper?"

TLDR: Not in >10 years (c.f. NumPy), but perhaps in the medium-term (2021-2022).

Long answer: I don't think we need to rush this, but good to have this issue up so that people are aware of how software is becoming a citable unit and worth putting effort into! See e.g. https://research-software.org/citation/ and the FORCE11 Software Citation Implementation Working Group at https://github.com/force11/force11-sciwg.

• An authorship policy. It's really important that we agree on this before doing any work a paper. See the Fatiando policy which was generated after long discussion and research into similar policies.

👍 Just want to point out CRediT – Contributor Roles Taxonomy too. We should acknowledge not just those who write software, but also the people that help us get funding, manage the project, etc. Also need to work on our diversity, let's try and avoid an 'all male' author paper 🙂

• Define a minimum viable product. What does PyGMT need to have in order to be fully usable? I'd say we're pretty close to having a usable alternative to Cartopy (if not there already). We don't necessarily need a "complete" product (when is software ever "done"?) but the package should be in usable state.

Subplots (#20) 😄 We've got most of the main GMT plotting modules covered, which is good if we want to just be a 'Cartopy' alternative.

But, if we want PyGMT to be more than a plotting library, then there's there's more work to do. Still a long way to go before reaching even half of all GMT modules (including supplementary ones) at https://docs.generic-mapping-tools.org/6.1/modules.html.

• Documentation! There is a lot of work to be done here and if we ignore the docs now and keep piling on features it willn be hard to do it later. I would suggest basing the structure on the Divio system. Thinking about this now will help us define the mvp.

I actually looked at the Divio documentation when making the changes at #643! We've made some good progress on the tutorial and gallery (how-to guide) front for the PyGMT v0.2.1 release (#665), and there's a backlog of new contributions to work through. Some more long-form presentation/workshop-style outreach material would certainly be welcome, just need to attend more conferences and get the name out!

• Decide on a journal. I see 2 options:

  • JOSS (pro = easy + free + actual code review | con = not as well known yet)
  • G³ (pro = highly visible in geoscience + GMT is already published there | con = expensive + no code review)

I do like the JOSS option with its GitHub issue based open review process. There's also many options at https://www.software.ac.uk/which-journals-should-i-publish-my-software that could be a good fit.

That said, a G³ paper is tempting. I've had my eye on Authorea for a while, which is a preprint server that allows direct submission to AGU journals (like G³) and allows for hosting data and code alongside figures. Still some rough edges (unsure how well they support third-party code like pygmt but certainly in the next year or two perhaps.

[leouieda]

TLDR: Not in >10 years (c.f. NumPy), but perhaps in the medium-term (2021-2022).

💯

But, if we want PyGMT to be more than a plotting library, then there's there's more work to do. Still a long way to go before reaching even half of all GMT modules (including supplementary ones)

I would challenge whether "wrapping GMT modules" is a metric we should have. From using PyGMT the past few months, I've come to think that simply wrapping GMT modules in functions/methods doesn't make for a good Python library. It may be powerful but it's not pleasant. But I agree 100% that it should be more than a plotting library.

Authorea

I've used it before but I kind of don't like a lot of aspects. I would push for writing the paper on GitHub in LaTeX or Markdown. We already work on this platform all the time anyway.

Also need to work on our diversity, let's try and avoid an 'all male' author paper

Yes! And this takes long term effort on our part. Real effort, not just having the door open but search for people to invite inside. GMT has had an open door for 30 years and that hasn't worked. Same with Numpy. Things only change when we actively push for it.

[MarkWieczorek]

I think that it would be great to have a scientific research paper that describes pygmt (like GMT). It is also great for a CV, and such articles are guaranteed to be highly cited. I did the same thing with pyshtools (also at G3), but in retrospect, writing such a paper is a lot of work, and given that the code is always changing, a static article isn't the best tool for someone trying to learn how to use the code.

What I would probably do is the following:

1. Focus on making the web documentation as complete as possible.
2. Add a couple of jupyter notebooks to show how to use key modules, and increase the number of examples in the gallery.
3. Write an advertisement in something like EOS, which is where the original GMT announcement was published. This could be as short as ~6 paragraphs, or something that is more like GRL in length.
4. Write a scientific article afterwards, which would be largely a cut-n-paste of the above.

I don't think that pygmt needs to reach parity with GMT, only that the quality of the documentation needs to be as good as it is for GMT.

[PaulWessel]

We will put in bucks for a G3 article for PyGMT in the NASA proposal we are submitting this week. If it gets funded then you have funds to pursue an Open publication with no access restrictions, like the GMT paper.

[weiji14]

Just putting down some notes from Day 1 after SciPy 2021, in particular after a Birds of a Feather session on PyOpenSci. From what I understand, going through the PyOpenSci process seems like a good way to:

1. Ensure that PyGMT meets established best practices in the Python/Open Science/Open Source ecosystem in terms of documentation/Continuous Integration/packaging/etc
2. Make us ready for a more thorough peer review process for publication in JOSS or G3 at a later date.

If people are keen, we could put in a pre-submission inquiry or proper submission at https://github.com/pyOpenSci/software-review. The review process takes place in a GitHub issue, and would take about 1-2 months (https://www.pyopensci.org/contributing-guide/open-source-software-peer-review/intro.html#the-peer-review-timeline), depending on how much we need to do. I/@meghanrjones could also find out more on the SciPy 2021 #pyopensci Slack channel this week to find out more if people think this is a good idea.

P.S. Geochemistry, Geophysics, Geosystems (G3) will be fully open access for all submissions after 8 Sep 2021. The price will drop from $3500 to $2750 according to https://www.agu.org/Publish-with-AGU/Publish/Author-Resources/Publication-fees 🎉

[leouieda]

@weiji14 I think this is a great idea! Leah and the rest of the pyOpenSci team are wonderful and there is a lot to gain from this. If submitting to JOSS, we can fasttrack the review process. And if going with G3, then the code will get reviewed (since the journal reviewers are unlikely to do this).

[weiji14]

Cool, sounds like a plan. I'll send in a pre-submission inquiry today (done at pyOpenSci/software-submission#43). Meanwhile, I've put up an editable document at https://hackmd.io/@pygmt/pyopensci for the full PyOpenSci review process, feel free anybody to edit it (let me know if you need access to hackmd). I will leave the document up for a week until Friday 30 Jul (UTC 12:00), depending on how the pre-submission inquiry goes, and then proceed to submit it to the PyOpenSci repo.

Edit: Haven't heard back from the pre-submission issue because the main editor is away on leave I think. Will report how things go once I hear back!

[weiji14]

So, we've just got accepted on PyOpenSci at pyOpenSci/software-submission#43 (comment) 🥳

What are people's thoughts on proceeding with a JOSS paper versus G3? I think the situation has changed quite a bit since last year (people moving countries, jobs, etc), so want to see what are people's priorities.

[liamtoney]

That's awesome news! Thanks for pushing that forward.

I feel that we may as well move forward w/ the JOSS paper, as the effort required to get that out seems relatively low.

Could crafting a G³ paper be an independent decision? I.e. do JOSS first and then make a call on G³ at a later date.

[weiji14]

Could crafting a G³ paper be an independent decision? I.e. do JOSS first and then make a call on G³ at a later date.

Need to double-check their policy, maybe @leouieda knows? I guess we could always follow GMT's lead and wait a few years or so before publishing a second, third or fourth paper 🤣 Oh, and another thing about $G^3$ is that we'll need to find $2750 somehow.

Actually, how about we vote on this @GenericMappingTools/pygmt-team. 🚀 for JOSS, and 🎉 for $G^3$. (You can also vote for both if undecided).

[leouieda]

Could crafting a G³ paper be an independent decision? I.e. do JOSS first and then make a call on G³ at a later date.

No, we can't. At least not unless there has been major changes to PyGMT that justifies a new publication. A JOSS paper is like any other, so submitting there means closing the doors on other journals until another very major PyGMT release (2.0 probably?).

Haven't cast a vote yet but here are some thoughts:

• JOSS:
  • Pros: Low effort (short paper + pyOpenSci review), diamond OA
  • Cons: Domain agnostic so not good for marketing, short paper format can make it hard for review boards to take it seriously (for example, in academic job apps / tenure / etc). Not always the case but we can't pretend it's not a possibility
• G³:
  • Pros: Helps get the word out. Longer paper in high profile journal = more points when being reviewed by other scientists.
  • Cons: High-ish effort (have to write a full paper), expensive, Wiley.

Thinking about those needing this as a career boost, I'd say we go for a longer form article. Don't get me wrong, I love JOSS and have published there. But the reality is most people don't yet take those papers as seriously as they should. I'm willing to put in major effort to write this if it's the way people want to go.

Another venue could be Journal of Open Research Software: More like JOSS but expects a longer form more traditional paper. APC is cheap.

As per the G³ APC, Liverpool has an agreement with Wiley so I can probably get this paid for by the university: https://authorservices.wiley.com/author-resources/Journal-Authors/open-access/affiliation-policies-payments/jisc-agreement.html

I'm happy to go with what you all think is best 🙂

[PaulWessel]

Also, @leouieda is co-PI on the GMT grant so there are funds that can be used to pay for any page charges.

[liamtoney]

Given the above justifications, including APC coverage, I've changed my vote. I am in the "needing a career boost" category. I also think the increased exposure / "impact" of G³ might be worth the added effort.

I will be able to put in intermediate effort on a G³ publication.

[seisman]

I also vote for a G3 publication because of its higher visibility to the geoscience community. My only concern is, is PyGMT ready for a static paper? Currently, PyGMT uses arguments like ["xaf+lLable", "WSen+tTITLE"], which is not pythonic and is likely to be changed in the near future.

[MarkWieczorek]

There is also the AGU open access journal Earth and Space Science, and I think that the topic fits with what they are looking for. There's nothing wrong with G3: I published the phystools paper there, but we chose this mostly because there weren't any other good choices at that time.

[leouieda]

@seisman all journal publications are “static”, including JOSS. Which is why we make them about a particular version of the software. Just like with GMT, when there is major progress and all arguments are fully Pythonic with new APIs, that could be marked by a new paper somewhere.

Thanks for sharing that journal @MarkWieczorek. Hadn’t seen it before but looks interesting.

[willschlitzer]

Will we have to/be expected to release v1.0 before a paper can be published? Is there an anticipated timeline for when this paper would be completed? If so, what are the remaining changes/features that need to be done before the paper (and/or v1.0 release).

Apologies in advance for any silly questions, I've never been involved in peer-reviewed work before.

[yvonnefroehlich]

I apologize for my delayed response!
Thanks at @weiji14 for pushing this forward! My vote for $G^3$ was / is based on its higher visibility and that GMT is already published there. Furthermore, I feel @leouieda is right and papers in JOSS are currently not taken equally seriously. However, this journal and the review process looks quite promising!
From a seismologist perspective, I feel, that the PyGMT release, for which the PyGMT paper is written, should include the two methods [ps]meca and [ps]coupe (see e.g. issue #2016 and issue #2019).

[maxrjones]

G³ or a similar journal is fine with me.

I would prefer to make some progress on more Pythonic syntax before publishing so that we can describe and provide examples of how complicated modifiers/directives will be handled in PyGMT. As a more concrete example, I think the ideal timing for a first paper would be after having a Pythonic API for the capabilities of one frequently used method (e.g., Figure.basemap) and all the common options (e.g., region, projection).

Given the balance between having a library closer to v1.0, the timing of the GMT grant for page fees, and the benefit of papers for career advancement, perhaps we should start writing even though we haven't met that mark? It would be quicker to revise a paper to reflect first steps on #1082 and the additions mentioned in #677 (comment) than it would be to start the manuscript after all the development prerequisites are finished. Also, that way we could publish quicker if the API modifications take too long to be included in the first paper.

[michaelgrund]

I'm also fine with G³. In contrast to @liamtoney I'm not in the "needing a career boost" category since I'm not working in academia anymore. However, I could also put in intermediate effort to the writing process in my spare time.

[willschlitzer]

@GenericMappingTools/pygmt-maintainers Is there any work that needs to be done/prioritized for this paper? I'm currently funemployed and would be happy to put in some time.

[PaulWessel]

I would strongly encourage the team to start working on a paper. While I do not have future NSF proposal planned I think we owe NSF a PyGMT paper given their support (of Leo) and more. As mentioned, we have funds for page charges so a G3 would be fine. IF it is a technical brief (like the GMT 6 paper) it will sail through. But it has to be written first.

[michaelgrund]

Agree with @PaulWessel. If we go with G3, what's the best platform to work as team on the paper? I saw that for all AGU journals there is support to use an official Latex template (https://www.agu.org/Publish-with-AGU/Publish/Author-Resources/Text-requirements) which is also available in overleaf (https://www.overleaf.com/gallery/tagged/agu-official#.WBukk-ErLus). So may overleaf be a solution to work with?

[seisman]

Is there any work that needs to be done/prioritized for this paper? I'm currently funemployed and would be happy to put in some time.

@willschlitzer Thanks for pushing it forward.

I agree with what @maxrjones said about the more Pythonic syntax in #677 (comment). A Pythonic API will definitely be a big change for PyGMT. I think we can still write the paper using the current PyGMT syntax (e.g., frame=["WSen", "xaf+lXLABEL", "yaf+lYLABEL"], but at least we should point out the future development direction about Pythonic syntax and give at least one example, just like what GMT does about the long-form options in their G3 paper.

If we go with G3, what's the best platform to work as team on the paper? I saw that for all AGU journals there is support to use an official Latex template (agu.org/Publish-with-AGU/Publish/Author-Resources/Text-requirements) which is also available in overleaf (overleaf.com/gallery/tagged/agu-official#.WBukk-ErLus). So may overleaf be a solution to work with?

Writing a paper using LaTeX may be challenging for people who're not familiar with LaTeX. I prefer to other simpler options like Google Docs, Markdown in a GitHub repository or HackMd which we're using for drafting release announcements.

[liamtoney]

My 2 cents are that Google Docs would work well for this paper. Agree that we can write it with the API as-is but be clear about the future aim of more Pythonic interface.

[seisman]

@GenericMappingTools/pygmt-team Let's have a vote for the platform we will work on writing the paper.

• 👍 for Google Docs
• ❤️ for Markdown save in a GitHub repository
• 🚀 for HackMd
• 🎉 for LaTeX on Overleaf
• 👀 for other options

Please note that no matter which platform we choose, we may still need to have a GitHub repository to store the scripts and images that will be included in the paper.

[seisman]

Based on the votes, I think we will go with HackMd.

I've created a template file for the PyGMT paper. I think everyone can read it and the @pygmt team members should be able to edit it.

Link to the PyGMT paper on HackMd: https://hackmd.io/@pygmt/rkmH1IsPs

[MarkWieczorek]

I know that I am late to vote: but what about just using the format that the journal expects for the submission? If it is Word, then Google docs, if it is latex, then overleaf. We will need to format equations, code, tables, and references, so if you need to translate to another format at the end, it might be a waste of time for someone.

[yvonnefroehlich]

I apologize that I missed that vote 🙁!
Thanks @willschlitzer for pushing this forward, and thanks @seisman for setting up the vote and the draft on HackMd!
Hm. If it is welcome that I contribute to the PyGMT paper, I feel I have to be added to the PyGMT Team HackMd 😉. I would be pleased to help 🙂.

[maxrjones]

I would be pleased to help 🙂.

Awesome! Can you sign up for an account on HackMD? It's easy to link to GitHub, but it's also possible to sign up using a separate email address. Then we can add you to the PyGMT team on HackMD.

[yvonnefroehlich]

I would be pleased to help 🙂.

Awesome! Can you sign up for an account on HackMD? It's easy to link to GitHub, but it's also possible to sign up using a separate email address. Then we can add you to the PyGMT team on HackMD.

Great 🙂!
Hm. I acctually already signed up for an accout on HackMD (see https://hackmd.io/@yvonnefroehlich) and it is linked to GitHub. Or do I have to change something else in the settings?

[seisman]

I would be pleased to help 🙂.

Awesome! Can you sign up for an account on HackMD? It's easy to link to GitHub, but it's also possible to sign up using a separate email address. Then we can add you to the PyGMT team on HackMD.

Great 🙂! Hm. I acctually already signed up for an accout on HackMD (see hackmd.io/@yvonnefroehlich) and it is linked to GitHub. Or do I have to change something else in the settings?

@yvonnefroehlich I've added you to the HackMD team. Now you should be able to edit it.

[yvonnefroehlich]

I would be pleased to help 🙂.

Awesome! Can you sign up for an account on HackMD? It's easy to link to GitHub, but it's also possible to sign up using a separate email address. Then we can add you to the PyGMT team on HackMD.

Great 🙂! Hm. I acctually already signed up for an accout on HackMD (see hackmd.io/@yvonnefroehlich) and it is linked to GitHub. Or do I have to change something else in the settings?

@yvonnefroehlich I've added you to the HackMD team. Now you should be able to edit it.

Thanks @seisman! Yes, now I am able to edit the document.

[MarkWieczorek]

Could you add me to the team as well? markwieczorek / @mrak

[liamtoney]

Thanks for setting this up, @seisman. Do we have a protocol for editing at this stage? For example, I could add in an outline of the "Community" section.

[seisman]

Could you add me to the team as well? markwieczorek / @mrak

Added. Edit: The written permission was temporarily revoked and will be considered after some internal discussions.

Thanks for setting this up, @seisman. Do we have a protocol for editing at this stage? For example, I could add in an outline of the "Community" section.

Feel free to make edits to the paper.