[Review] Citations page for Tax-Calculator

[chusloj]

This PR addresses #2470 regarding automatically adding citations to Tax-Calculator Jupyter Book docs.

Checklist:

• : Create and upload citations.bib and citations.md files (citations.md is used in the Jupyter Book docs).
• : Create launchd script that uses curl to scrape citations.bib from Tax-Calculator Zotero base.

[codecov]

Codecov Report

Merging #2473 into master will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2473   +/-   ##
=======================================
  Coverage   99.92%   99.92%           
=======================================
  Files          13       13           
  Lines        2582     2591    +9     
=======================================
+ Hits         2580     2589    +9     
  Misses          2        2           

Impacted Files	Coverage Δ
taxcalc/calculator.py	100.00% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ddc7232...8b23280. Read the comment docs.

[MattHJensen]

@chusloj just added a WIP tag. Please remove when this is ready for feedback. Thanks for your work on this!

[chusloj]

All tasks outlined here are finished.

For the citations:

• A citations.bib file has been created using the Tax-Calculator citations from Zotero.
• A bibliography has been added to the Jupyter Book documentation as zcitations.md.

The bibliography markdown file in Jupyter Book is called zcitations.md due to build issues related to sphinx-bibtex. In order to render the entire bibliography regardless of whether entries have been cited in the docs, the :all: flag from sphinx-bibtex is passed in zcitations.md to force the entire bibliography to build.

NOTE: When generating the Jupyter Book docs, the citations.bib file must have the author field filled in for every entry (or a similar field, i.e. institution in the case of reports). pybtex, the BibTeX parser underlying sphinx-bibtex, is very picky, so the docs will not recognize the :all: flag mentioned above.

---

For automation:

• A shell script has been created to pull new citations from Zotero using curl.
  • This shell script has been scheduled to run every night at 1 AM using launchd.
• An AUTOMATION.md documentation file has been created. The documentation is very similar to the one @Peter-Metz uses in the catalog_builder for PSLModels.org.

[chusloj]

NOTE:

In order to render the entire bibliography regardless of whether entries have been cited in the docs, the :all: flag from sphinx-bibtex is passed in zcitations.md to force the entire bibliography to build.

Previously, the :all: flag did not render in the docs, so a docs page was created which cited every entry in the bibliography, but the page containing those citations did not appear in the docs. Now that the :all: flag works, that docs page has been deleted.

[MattHJensen]

@chusloj, thanks for working on this!

Currently, it doesn't seem like the .md file will be reviewable before it goes live. Is that right? Also, it seems like the .bib file is updated in Tax-Calculator without a PR (although I'm not sure how the service has the credentials to do so), is that right? I am asking about these because It would be nice if any changes to the docs could be reviewed manually in a PR.

If the creation of such a PR with the full text of .bib and .md is automated, that is great. But it is also ok, for now, if the process requires manual steps in RELEASING.md as part of the X-Y-Z PR that happens at release time. (We will have a concerted effort to automate everything in RELEASING.md in the future as part of a move towards continuous deployment).

[chusloj]

Currently, it doesn't seem like the .md file will be reviewable before it goes live. Is that right? Also, it seems like the .bib file is updated in Tax-Calculator without a PR (although I'm not sure how the service has the credentials to do so), is that right?

Currently, the launchd script would just merge changes into master through a new branch, so no PR would opened for review. I know that GitHub's command line tool would probably be able to automatically open a PR. Also, the docs would have to be built locally in order to preview the citations page since the citations .md file only contains a call to load the .bib file but no actual citations.

Regarding the credentials, the launchd script would have to be run on a machine with permissions to merge to master.

For now, #2480 has been opened containing the .html file for the citations for preview.

[MattHJensen]

@chusloj, thanks again for this PR! I'm happy to proceed with this general approach and see how things go.

• I'd like PRs rather than direct commits so I can review changes before they are pushed to users.

• I noticed in Preview Tax-Calculator citations docs page #2480 that citations appear to be sorted alphabetically by last name. Could they be sorted by dates instead?

[chusloj]

@MattHJensen The citations page has been reformatted to sort citations by date from newest to oldest. This was done by sorting the bibliography by date during the curl call and stopping the Jupyter Book docs from performing any sorting. The change has been reflected in #2480.

[MattHJensen]

Thanks for the update @chusloj. Do you know why the date is goofy for this one? Are there any others / something systematic happening?

Alan D. Viard and Sita Nataraj Slavov. The Wrong Way to Make Social Security and Medicare More Progressive. Tax Notes, June 2020. URL: https://www.taxnotes.com/tax-notes-federal/social-welfare-taxation/wrong-way-make-social-security-and-medicare-more-progressive/2019/09/16/29wpk.

[chusloj]

@MattHJensen

Do you know why the date is goofy

Could you elaborate on what you mean by goofy? Do you mean why the date is not the first field in the citation?

[MattHJensen]

Could you elaborate on what you mean by goofy? Do you mean why the date is not the first field in the citation?

It is showing up as 2020 but it is a 2019 publication.

[chusloj]

@MattHJensen

It is showing up as 2020 but it is a 2019 publication.

Apologies, Zotero's scraping capabilities aren't perfect so some dates in the bibliography had to be edited. Citations file was updated and a new build preview has been pushed.

[chusloj]

This problem is addressed by #2494. Closing.