Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heavy slowdown in ExpandTemplates due to subtypes function #1438

Closed
epatters opened this issue Oct 17, 2020 · 2 comments · Fixed by #1452
Closed

Heavy slowdown in ExpandTemplates due to subtypes function #1438

epatters opened this issue Oct 17, 2020 · 2 comments · Fixed by #1452

Comments

@epatters
Copy link
Contributor

epatters commented Oct 17, 2020

The steps of the ExpanderPipeline are dispatched by iterating over the subtypes of this abstract type. The subtypes function is called for every Markdown element processed, meaning that it can easily be called hundreds or thousands of times during a run of Documenter.

However, under some circumstances (possibly involving generated functions or type metaprogramming, see #1261), the subtypes function is extremely slow:

julia> import Documenter

julia> @time subtypes(Documenter.Expanders.ExpanderPipeline);
  0.057625 seconds (92.08 k allocations: 5.367 MiB)

julia> @time subtypes(Documenter.Expanders.ExpanderPipeline);
  0.011200 seconds (2.44 k allocations: 724.125 KiB)

julia> import Catlab

julia> @time subtypes(Documenter.Expanders.ExpanderPipeline);
 40.238011 seconds (1.29 M allocations: 109.522 MiB, 0.18% gc time)

julia> @time subtypes(Documenter.Expanders.ExpanderPipeline);
  5.237196 seconds (29.11 k allocations: 4.189 MiB)

julia> @time subtypes(Documenter.Expanders.ExpanderPipeline);
  5.265813 seconds (29.11 k allocations: 4.189 MiB)

I figure this should be considered a bug in Julia itself and I'm going to file another issue upstream.

That being said, since the issue is currently severely impacting the build times for the docs of Catlab.jl and its dependers, I was hoping that a workaround could be implemented here. A simple workaround would move the call sort(subtypes(T); by = order) from inside dispatch to the body of the main function expand. That would ensure that subtypes is only called once during the ExpandTemplates step.

@MichaelHatherly
Copy link
Member

Thanks for tracking it down @epatters, would you be able to prepare a PR with that workaround and we can try get it sorted asap.

@epatters
Copy link
Contributor Author

Sure, I can prepare a PR. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants