-
Notifications
You must be signed in to change notification settings - Fork 6
Documentation Workflow Guide
With CICE or Icepack development, corresponding updates or modification to the CICE or Icepack documentation are required. Whenever you modify the model you should update documentation. The Consortium uses readthedocs.org to create online HTML and PDF documentation.
-
What is reStructuredText (RST)?
The CICE and Icepack documentation is written using reStructuredText (RST) markup language. ReStructuredText is a markup language, like HTML, markdown, or LaTeX. readthedocs.org is a tool for publishing RST documents in other formats such as HTML and PDF. Additional information about using RST and readthedocs.org are found in the sections below.
-
What is expected of me when changing the documentation?
Updated static PDF documentation will be generated for each new CICE or Icepack code release. However, the online "main" version of HTML or PDF documentation is considered a living document and will be updated regularly with regular code development workflow. Links to the "main" and static versions of CICE and Icepack documentation can be found in the Resource Index.
We expect that if you need to add or modify documentation that you will be able to modify the RST source files and generate HTML in order to review the HTML documentation. We will review the RST and HTML during a Pull Request to verify it is working properly and is consistent with the rest of the Consortium documentation format. When the Pull Request is successful, it will trigger a new documentation build for the main branch of the documentation. The new documentation build will create the HTML and PDF versions of CICE's documentation along with your updates.
In particular, it is important that you test out tables, equations, section references, figures, and/or citations in your contributed documentation as these can be particularly fiddly to get right.
-
Where are the documentation files kept?
The RST source files for generating HTML and PDF are stored in the repository under
/doc/source/.Links to the "main" and static versions of CICE and Icepack documentation can be found in the Resource Index.
The CICE-Consortium uses readthedocs.org to generate documentation for the CICE and Icepack models. As part of a Pull Request, model documentation will be build through readthedocs.org to verify that any contributions to the repositories correctly generate documentation.
However, as contributions to the model code and documentation are being developed it may be helpful to test the documentation before a Pull Request. We recommend that developers use readthedocs.org to test their contributions to the CICE documentation. This tool does not require external libraries to be built on each developer's personal computer and is free and easy to use. You can follow the steps below and also reference the readthedocs Getting Started guide available from readthedocs.org.
Follow the general CICE-Consortium Git Workflow Guide to clone the repository and create your personal fork for model modifications. Whenever you modify the model you should update documentation. We recommend you update the documentation on the same branch of your fork on which you are developing and testing code.
There are some important files you will need in order to correctly build the documentation. These should all be included automatically when you fork from the CICE-Consortium repositories:
- /doc/requirements.txt : This file is necessary to get the references and citations working properly by using sphinxcontrib-bibtex. This file should not need to be modified by developers generally.
- /doc/source/conf.py : Basic documentation information for the Consortium including template, etc. This file should not need to be modified by developers generally.
- /doc/source/zreferences.rst : required for the references to link properly. This file should not need to be modified by developers generally.
- /doc/source/master_list.bib : the master list of references cited in the documentation. This file may need to be modified by developers with documentation updates. This file is currently ordered sequentially from oldest to newest and alphabetically within a given year. To add references for your documentation, edit the master_list.bib file using the Articles and/or Books entries as examples for your addition(s). Please follow the format for ordering the date/alphabetization as well as including a URL with the document's DOI.
Open the RST file using a text editor and make the changes necessary. Note that from the User's Guide documentation (see link above) there is a hyperlink called "Show Source" on the left hand column that will show you the RST source code for the HTML you are viewing. This is a good way to see the syntax for tables, equations, linking references, labeling tables or figures, and correctly identifying documentation sections or subsections.
Here are some resources for using RST files:
-
RST tables - Note that tables can be tricky in Sphinx and we prefer using comma separated tables whenever possible.
When you're happy with the documentation you've generated, follow the standard Consortium Git Workflow Guide to do a Pull Request and make sure to note in the Pull Request Template that documentation has also been updated. The Read the Docs Github hook will automatically generate the HTML and PDF documentation once the Pull Request is opened. It can be particularly helpful if you include link(s) to the modified part(s) of the built documentation in the Pull Request Template. You can find the URL for your generated documentation at the bottom of the Pull Request page, by clicking on "Show all checks" and then "Details" next to the read-the-docs item.
If you start from a LaTeX (*.tex) document you will need to convert this to the RST format that Sphinx
requires. A handy tool to do this is Pandoc, which you
can install quickly and run from the command line.
Once Pandoc is installed, the basic command line syntax to convert a file is ::
$ pandoc NAMEIN.tex -f latex -t rst -s -ou NAMEOUT.rst
The NAMEOUT.rst file can be directly edited for Sphinx. Pandoc does a beautiful job of converting the text, equations, and many tables. However, equation numbering, section linking, references, figures, and some tables required more hands on care to be sure they render correctly.
Pandoc requires that the *.tex files be in utf-8 encoding. To easily do this open the *.tex
document in Emacs then do ctrl-x ctrl-m f and you will be prompted to enter encoding type. Just
type in utf-8 and hit enter. Then save with ctrl-x ctrl-s . You are done and the document can be
converted with Pandoc.
- Sign up for a free account at readthedocs.org
Select a username and password. These do not have to match your GitHub username and password, but having the same username can be simpler if the user chooses to do this. Below, USERNAME is a placeholder - you would need to replace this with your personal username.
- Connect your GitHub account
Click on your username in the upper right hand corner and select 'Settings' and then select 'Connected Services' to connect your GitHub account. This process will ask you to authorize a connection to readthedocs.org that allows for reading of information about and cloning of your repositories.
- Import your projects
Click on your username in the upper right hand corner and select 'My Projects'. Then click the 'Import a Project' green button. This will generate a list of repositories you are able to import. To add a repository click the + button. Once added and imported to readthedocs, the icon will change to an upward pointing arrow.
- Modify the project settings
Click on the project you are editing then click on the 'Admin' tab on the far right. The CICE-Consortium has found the following settings to be important for proper building.
Under 'Settings' modify and save the following:
- Name: USERNAME CICE (this is the local name of the repository on readthedocs.org)
- Repository URL:
https://github.com/USERNAME/CICE.git - Repository type: Git
- Description: (Add anything that would be useful to you to describe your fork.)
- Documentation type: Sphinx html
- Language: English
- Programming Language: Only Words
Under 'Advanced Settings' modify the following:
- Install Project: Unchecked box
- Requirements file: doc/requirements.txt (VERY IMPORTANT, see below)
- Default branch: readthedocs (whatever branch you are working on for development. If not set, this will default to main. When testing documentation on a development branch, you can set the branch to the name of the development branch and the "latest" version of the documentation will reflect the rst files that are in your development branch.)
- Default version: latest (what your documentation build will be called)
- Enable PDF build: Checked box
- Enable EPUB build: Checked box
- Privacy Level: Public (this is useful to keep public if you want to point to the tested documentation as part of a Pull Request)
- Python Interpreter: Python 2.x
Once you've committed and pushed changes to the documentation *.rst files on your personal development fork.
Go to your readthedocs.org site and then select your project "Overview". Whenever you commit to your fork
the documents will automatically build. There is also an option to "Build a Version". Choose "latest"
and then click the green "Build version" button.
You will automatically be taken to the "Builds" page with a list of recent documentation builds. The documentation build you just started will be listed as "Triggered" and then "Building". If the build is successful the status will change to "Passed"; if the build is not successful then the status will change to "Failed". You can click on the "Passed" or "Failed" text to get information about the build and what might be problematic. The time of the build attempt is also listed with the most recent build appearing at the top of the list.
To see the HTML you just successfully built, go to "Overview" and click on "latest" under versions. To see the PDF you just successfully built, go to "Downloads" and click on "latest PDF".
We recommend that you use readthedocs.org to test documentation. However, it is also possible to use Sphinx to build and test documentation. If you choose to follow this workflow, below are some tips for using Sphinx.
Sphinx must be installed once on each platform. See Sphinx or Installing Sphinx for details. Below are the commands for installing Sphinx on a mac laptop at the command line. Other platforms may require other steps::
$ sudo pip install --ignore-installed sphinx
$ sudo pip install --ignore-installed sphinxcontrib-bibtex
The CICE Consortium has used the following software to get successful Sphinx HTML builds, including linked references:
-
python 2.7.11
-
Sphinx (1.6.3)
-
sphinx-rtd-theme (0.1.9)
-
sphinxcontrib-bibtex (0.3.5)
-
sphinxcontrib-websupport (1.0.1)
As mentioned above, you will need the conf.py, zreferences.rst, and master_list.bib files that are part of the main branch and automatically included in your checkout. To use linked references you will need to have the sphinxcontrib-bibtex package as well.
Move into the /doc/ directory of your sandbox. Then execute the following command::
$ make clean
to get rid of old HTML files. Then execute::
$ make html
to build HTML into /build/html/ directory. It will also give you errors if there is a problem with the build that will help you figure out how you need to modify your RST files for a successful HTML build. Finally ::
$ open /build/html/FILE.html
Open the HTML on your browser for testing.
Generating a PDF is more complex and currently requires a two-step process. The generation will require recent versions of both LaTeX and Sphinx. From the /doc/ directory do the following::
$ make latex
$ cd build/latex
$ make
Then search for the *.pdf document created.
Clone the wiki, create a directory for the images if it does not already exist, copy the images into the directory, add paths to images in Wiki pages. Then git add, commit, and push; there are no pull requests for this process as there are with the code portion of the repository. Edit the wiki page online to add the image link, or do this in the clone before pushing.