Stingray Submission

[matteobachetti]

Submitting Author: @matteobachetti
All current maintainers: @dhuppenkothen, @mgullik, @jdswinbank, @matteolucchini1
Package Name: Stingray
One-Line Description of Package: A spectral-timing software package for astrophysical X-ray (and other) data
Repository Link: https://github.com/stingraysoftware/stingray

Version submitted: 2.1
EiC: @cmarmo
Editor: @hamogu
Reviewer 1: @taldcroft
Reviewer 2: @masonng-astro
Archive:
JOSS DOI: 10.21105/joss.07389
Version accepted: 2.1
Date accepted (month/day/year): 10/01/2024

---

Code of Conduct & Commitment to Maintain Package

• I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package after should it be accepted.
• I have read and will commit to package maintenance after the review as per the pyOpenSci Policies Guidelines.

Description

• Include a brief paragraph describing what your package does:

Stingray is a Python library for "spectral timing", i.e. time-series analysis techniques that can be used to study how variability changes or correlates between different energy bands/wavelengths. It is Astropy-affiliated, and with an ever growing user base now comprising hundreds of researchers around the globe.

Scope

• Please indicate which category or categories.
  Check out our package scope page to learn more about our
  scope. (If you are unsure of which category you fit, we suggest you make a pre-submission inquiry):

  • Data retrieval
  • Data extraction
  • Data processing/munging
  • Data deposition
  • Data validation and testing
  • Data visualization¹
  • Workflow automation
  • Citation management and bibliometrics
  • Scientific software wrappers
  • Database interoperability

Domain Specific

• Geospatial
• Education

Community Partnerships

If your package is associated with an
existing community please check below:

• Astropy:My package adheres to Astropy community standards
• Pangeo: My package adheres to the Pangeo standards listed in the pyOpenSci peer review guidebook

• For all submissions, explain how and why the package falls under the categories you indicated above. In your explanation, please address the following points (briefly, 1-2 sentences for each):

This package is mostly focused on the analysis of new data from high-energy missions. We added data validation and testing because it can be used as a quick look tool to point out possible anomalies in observations (e.g. solar or other background flares).

• Who is the target audience and what are scientific applications of this package?

The target audience is principally researchers and students of X-ray and multi-wavelength astronomy.
This package fills a void for a free and open source (spectral-)timing package for X-ray astronomy. XRONOS, formerly maintained by HEASARC, is currently unmaintained, and the analysis of high-energy timeseries is done mostly with proprietary software or mission-specific packages. We provide a Python package that eases the learning curve for newcomers, also thanks to extensive tutorials based on Jupyter notebooks, and provides experts with a powerful, robust library for their analysis. We provide software to analyze astronomical time series and do a number of things, including periodicity searches, time lag calculations, covariance spectra, power spectral modeling.

• Are there other Python packages that accomplish the same thing? If so, how does yours differ?

Our package is arguably the most well-known Python package for X-ray spectral timing.

• If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted:

#195

Technical checks

For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.
• uses an OSI approved license.
• [ x contains a README with instructions for installing the development version.
• includes documentation with examples for all functions.
• contains a tutorial with examples of its essential functions and uses.
• has a test suite.
• has continuous integration setup, such as GitHub Actions CircleCI, and/or others.

Publication Options

• Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Checks

• The package has an obvious research application according to JOSS's definition in their submission requirements. Be aware that completing the pyOpenSci review process does not guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS.
• The package is not a "minor utility" as defined by JOSS's submission requirements: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria.
• The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/. We have a previous JOSS paper describing the package. Is it acceptable to have a new paper, including the many improvements made in the last five years?
• The package is deposited in a long-term repository with the DOI:

Note: JOSS accepts our review as theirs. You will NOT need to go through another full review. JOSS will only review your paper.md file. Be sure to link to this pyOpenSci issue when a JOSS issue is opened for your package. Also be sure to tell the JOSS editor that this is a pyOpenSci reviewed package once you reach this step.

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.

• Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.

Confirm each of the following by checking the box.

• I have read the author guide.
• I expect to maintain this package for at least 2 years and can help find a replacement for the maintainer (team) if needed.

Please fill out our survey

• Last but not least please fill out our pre-review survey. This helps us track
  submission and improve our peer review process. We will also ask our reviewers
  and editors to fill this out.

P.S. Have feedback/comments about our review process? Leave a comment here

Editor and Review Templates

The editor template can be found here.

The review template can be found here.

Footnotes

1. Please fill out a pre-submission inquiry before submitting a data visualization package. ↩

[cmarmo]

Editor in Chief checks

Hi there! I'm Chiara and I am taking care of the Editor in chief role for the next three months.
Thank you for submitting your package for pyOpenSci review.
Below are the basic checks that your package needs to pass to begin our review.
If some of these are missing, we will ask you to work on them before the review process begins.

Please check our Python packaging guide for more information on the elements
below.

• Installation The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g. conda-forge, bioconda).
  • The package imports properly into a standard Python environment import package.
• Fit The package meets criteria for fit and overlap.
• Documentation The package has sufficient online documentation to allow us to evaluate package function and scope without installing the package. This includes:
  • User-facing documentation that overviews how to install and start using the package.
  • Short tutorials that help a user understand how to use the package and what it can do for them.
  • API documentation (documentation for your code's functions, classes, methods and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
• Core GitHub repository Files
  • README The package has a README.md file with clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
  • Contributing File The package has a CONTRIBUTING.md file that details how to install and contribute to the package.
  • Code of Conduct The package has a CODE_OF_CONDUCT.md file.
  • License The package has an OSI approved license.
    NOTE: We prefer that you have development instructions in your documentation too.
• Issue Submission Documentation All of the information is filled out in the YAML header of the issue (located at the top of the issue template).
• Automated tests Package has a testing suite and is tested via a Continuous Integration service.
• Repository The repository link resolves correctly.
• Package overlap The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly.
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

• Initial onboarding survey was filled out
  We appreciate each maintainer of the package filling out this survey individually. 🙌
  Thank you authors in advance for setting aside five to ten minutes to do this. It truly helps our organization. 🙌

---

---

Editor comments

Stingray is in excellent form! 💪
Congratulations!
I am going to check the availability of our Astropy editors and we will be back to you shortly.

[cmarmo]

Hello @matteobachetti I am pleased to announce that @hamogu will take care of your submission as editor.
Thank you Moritz!
I'm wishing to all of you a nice review process! :)

[hamogu]

Editor response to review:

---

Editor comments

👋 Hi @taldcroft and @masonng-astro! Thank you for volunteering to review
for pyOpenSci!

Please fill out our pre-review survey

Before beginning your review, please fill out our pre-review survey. This helps us improve all aspects of our review and better understand our community. No personal data will be shared from this survey - it will only be used in an aggregated format by our Executive Director to improve our processes and programs.

• reviewer 1 survey completed.
• reviewer 2 survey completed.
• reviewer 3 (if applicable)

The following resources will help you complete your review:

1. Here is the reviewers guide. This guide contains all of the steps and information needed to complete your review.
2. Here is the review template that you will need to fill out and submit
   here as a comment, once your review is complete.

Please get in touch with any questions or concerns! Your review is due:

Reviewers: Tom Aldcroft and Mason Ng
Due date: Sept 6th, 2024

[hamogu]

We have a previous JOSS paper describing the package. Is it acceptable to have a new paper, including the many improvements made in the last five years?

It's up to JOSS, not us, how exactly they handle that. Yo save everyone time, JOSS will generally accept our review as theirs, because there is really no need for a second set of reviewers that do essentially the same checks that our reviewers just did. However the final decision to accept into JOSS is up to JOSS.

So, I think there are two ways this can be done, at out choice:

• Either you contact JOSS and ask for their opinion. (In that case, I would pause this review until you hear back from them.)
• Or you write the paper.md as you would like to have it, and we review the code and paper as we normally do and pass it to JOSS once our review is done. (In that case, I would pause this review until you've written the paper.md so that our reviewers can actually review the paper.)

Option 2 is faster (you don't have to wait for the JOSS response first), but carries the risk that the JOSS editor decides not to publish it later.

Note that JOSS says: "we also review submissions that are significant contributions made to existing packages".
Based on this, I suggest option 2 (you add a 'paper.md` here before we review) and I will specifically instruct our reviewers to comment on if they judge the development in the last 5 years to be a "significant contribution".

[matteobachetti]

@hamogu we are working on the JOSS paper draft here: StingraySoftware/stingray#829
The text will not change much, we are working mostly on the author list and acknowledgements.
The new paper is paper2.md, and the bibliography joss2.bib (because we have the 2019 paper in the same directory)

[hamogu]

Just let me know when the paper is ready and I'll let the reviewers know to start their review. In the meantime, I'm emailing reviewers to find people to do the review when you are ready.

[matteobachetti]

@hamogu the paper is ready, but I'm still missing a couple OrcIDs and affiliations. Does it have to be merged or the link to the PR is sufficient?

[hamogu]

Just letting you know that I'm in the process of looking for reviewers; both I and some potential reviewers are on summer travel, but I'm hoping to get you two reviewers soon to get this started.

[hamogu]

Just letting you know that I'm still looking for reviewers. Seems that everyone is on vacation right now...

[matteobachetti]

Understandable 😅
No worries!

[hamogu]

👋 Hi @taldcroft and @masonng-astro!
Thanks for agreeing to do a review of stingray for us. Generic instructions (pre-review survey, etc.) are above: #201 (comment)

In addition, note that this package is also submitting to JOSS and JOSS will accept our review and not do a separate review. However, stingray already had a JOSS publication a few years ago. So, it would be very helpful to note in your review if you deem the new features added since then as new "substantial contribution" that merits a new JOSS publication. You can find the paper draft for JOSS in StingraySoftware/stingray#829

Please reach out to me if you have any questions!

[hamogu]

👋 Hi @taldcroft and @masonng-astro! I'm hoping to get your reviews for this package soon! If you have any problems or need help or guidance in how to do a review, please reach out to me any time!

[matteobachetti]

Thanks for the ongoing reviews!
FYI, I've started addressing comments on readme/docs at StingraySoftware/stingray#842

[taldcroft]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• A statement of need clearly stating problems the software is designed to solve and its target audience in README.
• Installation instructions: for the development version of the package and any non-standard dependencies in README.
• Vignette(s) demonstrating major functionality that runs successfully locally.
• Function Documentation: for all user-facing functions.
• Examples for all user-facing functions.
• Community guidelines including contribution guidelines in the README or CONTRIBUTING.
• Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a pyproject.toml file or elsewhere.
  • This information is in setup.cfg

Readme file requirements
The package meets the readme requirements below:

• Package has a README.md file in the root directory.

The README should include, from top to bottom:

• The package name
• Badges for:
  • Continuous integration and test coverage,
  • Docs building (if you have a documentation website),
  • A repostatus.org badge,
    • Not included. I somewhat question the utility of this badge since it seems to rely
      on the package maintainers to keep it up to date. An abandoned project is not
      likely to be marked as such.
  • Python versions supported,
    • Looks like Py 3.8 to 3.12 are tested, but I do not see this in the install docs.
  • Current package version (on PyPI / Conda).

NOTE: If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the a badge for pyOpenSci peer-review will be provided upon acceptance.)

• Short description of package goals.
• Package installation instructions
• Any additional setup required to use the package (authentication tokens, etc.)
• Descriptive links to all vignettes. If the package is small, there may only be a need for one vignette which could be placed in the README.md file.
  • Brief demonstration of package usage (as it makes sense - links to vignettes could also suffice here if package description is clear)
• Link to your documentation website.
• If applicable, how the package compares to other similar packages and/or how it relates to other packages in the scientific ecosystem.
• Citation information

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole.
Package structure should follow general community best-practices. In general please consider whether:

• Package documentation is clear and easy to find and use.
• The need for the package is clear
• All functions have documentation and associated examples for use
• The package is easy to install

Functionality

• Installation: Installation succeeds as documented.
• Functionality: Any functional claims of the software been confirmed.
  • I do not have the domain knowledge or resources to confirm the functional claims of
    the software with regards to X-ray spectral timing. The extensive collection of
    notebooks should allow the motivated scientist to evaluate and understand the
    results from this package.
• Performance: Any performance claims of the software been confirmed.
  • Documentation makes no specific performance claims.
• Automated tests:
  • All tests pass on the reviewer's local machine for the package version submitted by the author. Ideally this should be a tagged version making it easy for reviewers to install.
  • Tests cover essential functions of the package and a reasonable range of inputs and conditions.
• Continuous Integration: Has continuous integration setup (We suggest using Github actions but any CI platform is acceptable for review)
• Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.
  A few notable highlights to look at:
  • Package supports modern versions of Python and not End of life versions.
  • Code format is standard throughout package and follows PEP 8 guidelines (CI tests for linting pass)

For packages also submitting to JOSS

• The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

• A short summary describing the high-level functionality of the software
• Authors: A list of authors with their affiliations
• A statement of need clearly stating problems the software is designed to solve and its target audience.
• References: With DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

• The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 5

---

Review Comments

I am reviewing this package as an (inactive) astronomer and active Astropy developer.

From a scientific perspective it appears that the stingray package fills a distinct
need for modern open source Python-based X-ray spectral timing tools. It is being widely
used in the X-ray timing community, with 170 citations to the 2019 ApJ paper. In recent
years it is seeing around 35 citations per year.

From a software perspective this checks all the boxes (literally) for a package that
meets current standards for a well-developed, documented, and maintained package. The
test coverage is excellent.

I ran all the notebooks and found a couple of problems (documented as issues). This
highlights the need for StingraySoftware/stingray#802 to be
worked, but overall things are looking OK and those notebook issues are not a blocker
for accepting the package.

I opened one issue to document how to download the tutorial notebooks that form most of
the package functional documentation. This is a simple docs update and I would like to
see this done for accepting the package.

[matteobachetti]

Update:

• Responses to pyOpenSci review StingraySoftware/stingray#842 addresses issues with documentation and package infrastructure, including minimum Python versions, badges and so on (under review).
• Address PyOpenSci comments StingraySoftware/notebooks#103 addresses the issues with notebooks (ongoing)
• New JOSS paper draft StingraySoftware/stingray#829 addresses comments to the JOSS paper (ongoing)

[matteobachetti]

@masonng-astro I have a similar setup to yours (Macbook pro with M1 processor), but I couldn't reproduce your bugs 🆘

The failing test on unsorted data is particularly weird, as it does not seem to rely on any specific dependency other than the default ones. There is a vague chance that it is related to the numba installation, as I use numba to check for sortedness, but it should also revert to standard numpy if numba is not installed or it fails.

I could also install UltraNest without issues. I wonder if you have conflicting versions of some dependencies? From the list of libraries in your error, there seems to be some kind of clash between HEASOFT-installed libraries and the standard Python path. Is it possible?

[taldcroft]

FWIW as noted in my review, tests passed for me on Mac Silicon and Python 3.12. I created a fresh conda environment (using conda defaults channel) for Python 3.12 and then pip installed everything else.

[taldcroft]

With StingraySoftware/stingray#842 now merged, I have updated my review to recommend approving this package!

[hamogu]

@masonng-astro Could you have a look if your concerns have been resolved or if you can provide a more detailed look at why it fails? It seems that @matteobachetti tried to reproduce your failure, but failed to do so. Of course, we don't want to accept a failing package, but it's also hard to figure out what went wrong. 😕

Once you think it's ready to be accepted, please edit the post with your review and tick the box on "I recommend approving this package." and leave. short message here (GH will inform me when you post a new comment, but not when you edit an existing one).

[masonng-astro]

@masonng-astro I have a similar setup to yours (Macbook pro with M1 processor), but I couldn't reproduce your bugs 🆘

The failing test on unsorted data is particularly weird, as it does not seem to rely on any specific dependency other than the default ones. There is a vague chance that it is related to the numba installation, as I use numba to check for sortedness, but it should also revert to standard numpy if numba is not installed or it fails.

I could also install UltraNest without issues. I wonder if you have conflicting versions of some dependencies? From the list of libraries in your error, there seems to be some kind of clash between HEASOFT-installed libraries and the standard Python path. Is it possible?

Sorry for my late response - I did eventually manage to hunt down the issue and it boiled down to a weird esoteric mix of architectures in my Anaconda installation which I never encountered until now! Apologies for holding up the review. I'll go ahead and approve now. I did check everything else and all is good on my end!

[hamogu]

🎉 Stingray has been approved by pyOpenSci! Thank you @matteobachetti for submitting stingray and many thanks to @taldcroft and @masonng-astro for reviewing this package! 😸

Author Wrap Up Tasks

There are a few things left to do to wrap up this submission:

• Activate Zenodo watching the repo if you haven't already done so.
• Tag and create a release to create a Zenodo version and DOI.
• Add the badge for pyOpenSci peer-review to the README.md. The badge should be [![pyOpenSci Peer-Reviewed](https://pyopensci.org/badges/peer-reviewed.svg)](https://github.com/pyOpenSci/software-review/issues/issue-number).
• Please fill out the post-review survey. All maintainers and reviewers should fill this out.

It looks like you would like to submit this package to JOSS. Here are the next steps:

• Once the JOSS issue is opened for the package, we strongly suggest that you subscribe to issue updates. This will allow you to continue to update the issue labels on this review as it goes through the JOSS process.
• Login to the JOSS website and fill out the JOSS submission form using your Zenodo DOI. When you fill out the form, be sure to mention and link to the approved pyOpenSci review. JOSS will tag your package for expedited review if it is already pyOpenSci approved.
• Wait for a JOSS editor to approve the presubmission (which includes a scope check).
• Once the package is approved by JOSS, you will be given instructions by JOSS about updating the citation information in your README file.
• When the JOSS review is complete, add a comment to your review in the pyOpenSci software-review repo here that it has been approved by JOSS. An editor will then add the JOSS-approved label to this issue.

🎉 Congratulations! You are now published with both JOSS and pyOpenSci! 🎉

Editor Final Checks

Please complete the final steps to wrap up this review. Editor, please do the following:

• Make sure that the maintainers filled out the post-review survey
• Invite the maintainers to submit a blog post highlighting their package. Feel free to use / adapt language found in this comment to help guide the author.
• Change the status tag of the issue to 6/pyOS-approved6 🚀🚀🚀.
• Invite the package maintainer(s) and both reviewers to slack if they wish to join.
• If the author submits to JOSS, please continue to update the labels for JOSS on this issue until the author is accepted (do not remove the 6/pyOS-approved label). Once accepted add the label 9/joss-approved to the issue. Skip this check if the package is not submitted to JOSS.
• If the package is JOSS-accepted please add the JOSS doi to the YAML at the top of the issue.

---

If you have any feedback for us about the review process please feel free to share it here. We are always looking to improve our process and documentation in the peer-review-guide.

[hamogu]

@matteobachetti We at pyOpenSci would love to highlight stingray in a blog post similar to what these packages have done: pandera, movingpandas (We have an md template that you could use as a guide when creating your post.)
but I know that you've written a lot about Stingray before, and this is certainly not a requirement!

I also want to invite all three of you to the PyOpenSci slack - there is a very active community that discusses best practices in packaging, testing , and administration of Python packages. Personally, I found a lot of those discussions very insightful and useful, thought I myself have never contributed by posting - I just use it as a resource to look at. If you'd like to join, let me know which email address I should invite!

[matteobachetti]

@hamogu I opened a PR with the badge, and filled in the form. Later on I will open the JOSS issue. Thanks a lot!

[matteobachetti]

@matteobachetti We at pyOpenSci would love to highlight stingray in a blog post similar to what these packages have done: pandera, movingpandas (We have an md template that you could use as a guide when creating your post.) but I know that you've written a lot about Stingray before, and this is certainly not a requirement!

I think it's a good idea, I have some matters to deal with in the next few weeks but I'll try to do it. Thanks!

I also want to invite all three of you to the PyOpenSci slack - there is a very active community that discusses best practices in packaging, testing , and administration of Python packages. Personally, I found a lot of those discussions very insightful and useful, thought I myself have never contributed by posting - I just use it as a resource to look at. If you'd like to join, let me know which email address I should invite!

Sent in private message!

[hamogu]

I see that there is a JOSS DOI now, so I've updated all the information and will close this issue. Thanks everyone!