Matrices not rendered in sphinx documentation

[briandesilva]

The notebook 2_original_paper.ipynb contains multiple matrices typeset in LaTeX, but they are not rendered in the sphinx documentation.

The matrices were made using the bmatrix environment. From what I have read online, mathjax should be compatible with bmatrix, and we list mathjax as an extension in our configuration file, so I'm having trouble figuring out where the problem stems from. I couldn't find anything relevant in the nbexamples documentation either. @Ohjeah, do you have any insights into what the problem might be?

[Ohjeah]

Some notes while I am investigating this:

Going down the rabbit hole, the dependency chain of packages involved in rendering the notebook is: sphinx -> sphinx-nbexamples -> nbconvert.

In my local environment, sphinx-build -b html . build fails with lots of errors:

sphinx-build -b html . build
Running Sphinx v2.4.4
making output directory... done
Creating file /Users/mq/repo/github/dynamicslab/pysindy/docs/api/pysindy.rst.
Creating file /Users/mq/repo/github/dynamicslab/pysindy/docs/api/pysindy.differentiation.rst.
Creating file /Users/mq/repo/github/dynamicslab/pysindy/docs/api/pysindy.feature_library.rst.
Creating file /Users/mq/repo/github/dynamicslab/pysindy/docs/api/pysindy.optimizers.rst.
Creating file /Users/mq/repo/github/dynamicslab/pysindy/docs/api/pysindy.utils.rst.
Scaling examples/images/1_feature_overview_2.png to thumbnail examples/images/thumb/gallery_examples_1_feature_overview.ipynb_thumb.png
Scaling examples/images/2_original_paper_10.png to thumbnail examples/images/thumb/gallery_examples_2_original_paper.ipynb_thumb.png
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 14 source files that are out of date
updating environment: [new config] 14 added, 0 changed, 0 removed
reading sources... [100%] index
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:36: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:38: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:50: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:52: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:85: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:79: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:82: WARNING: Unexpected indentation.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:86: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:87: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:96: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:97: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:98: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:107: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:108: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:109: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:119: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:120: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:121: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:136: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:138: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:146: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:148: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:170: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:172: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:300: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:302: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:382: WARNING: Unexpected indentation.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:377: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:385: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:454: WARNING: Unexpected indentation.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:451: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:457: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:540: WARNING: Unexpected indentation.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:537: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:544: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:545: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:847: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:848: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:849: WARNING: Definition list ends without a blank line; unexpected unindent.
looking for now-outdated files... none found
pickling environment... /Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:864: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:866: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:883: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:875: WARNING: Bullet list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:877: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:880: WARNING: Unexpected indentation.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:882: WARNING: Bullet list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:884: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:885: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:889: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:891: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:895: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:897: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:1053: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:1054: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:1055: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:1063: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/2_original_paper.rst:1065: WARNING: Definition list ends without a blank line; unexpected unindent.
done
checking consistency... done
preparing documents... /Users/mq/repo/github/dynamicslab/pysindy/docs/examples/feature_overview.rst: WARNING: document isn't included in any toctree
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/papers/index.rst: WARNING: document isn't included in any toctree
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/papers/sindy_paper.rst: WARNING: document isn't included in any toctree
/Users/mq/repo/github/dynamicslab/pysindy/docs/examples/sindy_paper.rst: WARNING: document isn't included in any toctree
done
writing output... [100%] index
generating indices...  genindex/Users/mq/repo/github/dynamicslab/pysindy/pysindy/pysindy.py:docstring of pysindy.pysindy.SINDy.fit:27: WARNING: 'any' reference target not found: SINDy.optimizer
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/pysindy.py:docstring of pysindy.pysindy.SINDy.fit:27: WARNING: 'any' reference target not found: unbias=True
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/feature_library/polynomial_library.py:docstring of pysindy.feature_library.polynomial_library.PolynomialLibrary:8: WARNING: 'any' reference target not found: x[i] ** k
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/feature_library/polynomial_library.py:docstring of pysindy.feature_library.polynomial_library.PolynomialLibrary:11: WARNING: 'any' reference target not found: degree
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/feature_library/polynomial_library.py:docstring of pysindy.feature_library.polynomial_library.PolynomialLibrary:11: WARNING: 'any' reference target not found: x[1] ** 2
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/feature_library/polynomial_library.py:docstring of pysindy.feature_library.polynomial_library.PolynomialLibrary:11: WARNING: 'any' reference target not found: x[0] * x[2] ** 3
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/optimizers/sindy_optimizer.py:docstring of pysindy.optimizers.sindy_optimizer.SINDyOptimizer:3: WARNING: 'any' reference target not found: _unbias
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/optimizers/sindy_optimizer.py:docstring of pysindy.optimizers.sindy_optimizer.SINDyOptimizer:7: WARNING: 'any' reference target not found: optimizer
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/optimizers/sindy_optimizer.py:docstring of pysindy.optimizers.sindy_optimizer.SINDyOptimizer:7: WARNING: 'any' reference target not found: fit_intercept
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/optimizers/sindy_optimizer.py:docstring of pysindy.optimizers.sindy_optimizer.SINDyOptimizer:7: WARNING: 'any' reference target not found: normalize
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/optimizers/sindy_optimizer.py:docstring of pysindy.optimizers.sindy_optimizer.SINDyOptimizer:11: WARNING: 'any' reference target not found: optimizer=STLSQ(alpha=0.1)
/Users/mq/repo/github/dynamicslab/pysindy/pysindy/optimizers/sindy_optimizer.py:docstring of pysindy.optimizers.sindy_optimizer.SINDyOptimizer:11: WARNING: 'any' reference target not found: unbias=True
 py-modindexdone
highlighting module code... [100%] pysindy.utils.base
writing additional pages...  search/Users/mq/.miniconda/envs/pysindy-dev/lib/python3.8/site-packages/sphinx_rtd_theme/search.html:20: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert a <script> tag directly in your theme instead.
  {{ super() }}
done
copying images... [100%] examples/images/sindy_paper_10.png
copying downloadable files... [100%] examples/sindy_paper.ipynb
copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 73 warnings.

The HTML pages are in build.

I think there are some newline in the multiline latex code inside the jupyter notebook which are the root cause:

Inside the notebook we have

\begin{equation}
    \frac{d}{dt}x(t) = f(x(t).
\end{equation}

Invoking sphinx, the notebook is translated to an intermediate rst file (prob. managed by sphinx-nbexamples calling nbconvert). The latex equation is converted to

:raw-latex:`\begin{equation}
    \frac{d}{dt}x = f(x) = \begin{bmatrix} f_1(x)\\f_2(x) \end{bmatrix} = \begin{bmatrix}1 - x_1 + 3x_1x_2 \\ x_2^2 - 5x_1^3 \end{bmatrix}
\end{equation}`

breaking rst roles. :role: should be a one-liner AFAIK. Removing the newlines (\begin{equation}\frac{d}{dt}x(t) = f(x(t).\end{equation}) fixes the issue as a workaround.

I can reproduce this with jupyter nbconvert --to rst 2_original_paper.ipynb.

I guess this is a known limitation of nbconvert (jupyter/nbconvert#291).

[briandesilva]

I went through and removed the newlines in #63. Let me know if you think what I did there will fix the issue. I ran jupyter nbconvert --to rst 2_original_paper.ipynb on the updated notebook and all the LaTex was formatted like the following example in the resulting rst file:

.. raw:: latex

   \begin{equation} \frac{d}{dt}x(t) = f(x(t)). \end{equation}

[Ohjeah]

It works for me locally, but did not fix the issue online. What's your pandoc version?

[briandesilva]

I've got a fairly old version: 1.19.2.4.

After updating to 2.9.2.1 and running nbconvert, all math within the equation environment gets converted to this format:

:raw-latex:`\begin{equation} \frac{d}{dt}x(t) = f(x(t)). \end{equation}`

The problem may be with the equation environment. If I use $$ instead of \begin{equation} and \end{equation}, the math is instead converted to

.. math::  \frac{d}{dt}x(t) = f(x(t)).

I've created another PR using $$, #64.

[briandesilva]

It looks like #64 resolved the problem, so I'm closing this issue.

Note that when $$ is used, it looks like you can include newlines in math equations. I'm not going to go back and reformat everything right now, but it's useful to know for future jupyter notebook examples.