Docs now take a very long time to build

[epatters]

Since merging #219, there has been a massive jump in the build time for the docs, from ~10 min to one hour, as reported here. I have no idea what is causing this or what to do about it.

[epatters]

By setting the environment variable JULIA_DEBUG=all and building the docs locally, I can see roughly how much time Documenter.Expanders is spending on each file. It turns out that most of the time is spent not in the API docs but in Markdown files generated by Literate.jl. Most of these files are about manipulating and visualizing wiring diagrams, which has no intersection with #219 and involves no @generated code. This makes the sudden drastic slowdown all the more mysterious.

[olynch]

The intersection with #219 is that graphics uses the new property graphs...

I'll write some benchmarks for different ACSet operations (which is probably a good idea in any case), and we'll see if we can speed them up.

[olynch]

Or actually, I'll do some profiling first on the API docs.

[jpfairbanks]

Performance is one of my main concerns with ACSets right now. If they exercise the compiler in a big way, then we need to get involved with the Julia compiler team to get that pain point exposed before the 1.6 feature freeze or we will be living with slow Catlab for ~2 years on LTS Julia.

[olynch]

I have a hard time believing that the slow build is the compiler's fault; it must be a problem with my generated code. The only thing is, when I try to build the docs on my computer so that I can profile the build process, I get a SEGFAULT. I set up a docker container just like the docs.yml file, so I'm not sure what I'm doing wrong...

[epatters]

Try removing the ~/.julia/compiled directory. Occasionally I've had to do this to resolve segfaults that seem to be related to caching problems.

[epatters]

Owen, thanks a lot for investigating. Based on my experiments last night, my current best guess is that there is a bad interaction involving the new ACSets implementation, the Julia compiler, and the specific evaluation logic used by Documenter.jl. To summarize the above, here's what I'm seeing:

• There is no appreciable change to the runtime of test suite (8-10 min on GitHub CI)
• There is a large increase in the runtime of the docs, from ~10 min to 1 hr
• Much of the docs build time is spent running the Literate.jl notebooks
• Many of the notebooks should not exercise the C-sets code at all (e.g. the TikZ wiring diagrams) but still execute very slowly

[olynch]

If it remade the @generated code every time it ran, that might do it; I definitely was prioritizing clarity over performance in the code that generates the functions, and there are a couple things that are quadratic that don't necessarily need to be. But they are quadratic in an $n$ that never gets above 10 in our code, so I thought it didn't really matter, especially because they should only run once, and the inner loop of the function that makes it technically quadratic is really fast. So I don't think that that is the problem...

To sum up, I'm pretty happy to blame the Documenter.jl logic.

As a work-around, I wonder if we could take the file, do some basic source transformations so that each line spits its output (text, svg or png) to a file whose name was the name of that line, run that new file through the regular julia compiler, and then make a markdown file where the julia code was replaced by something that just includes the output.

[epatters]

I agree, I don't think there's anything obviously wrong with the code you @generate. As you can see from the linked issue, I've reported to Documenter.jl. I'm hopeful that with their help we can get to the bottom of it.

[epatters]

I think I discovered the problem or a large part of it. Issues have been filed upstream at JuliaDocs/Documenter.jl#1438 and JuliaLang/julia#38079.

[EricForgy]

Wild guess:

Is there some type piracy going on in Catlab? The issue JuliaDocs/Documenter.jl#1438 reminds me of the kind of thing you might see with precompilation invalidations that Tim et al have been working on.

Maybe SnoopCompile.jl might help?

[epatters]

Thanks for the pointer, that could be helpful for debugging. My current guess is that it has something to do with generated functions but I really don't know. First, I'm going to hope that the experts at JuliaLang can weigh in :)