Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache the source text for Base and update guidelines for using Revise #24120

Merged
merged 2 commits into from
Oct 19, 2017

Conversation

timholy
Copy link
Member

@timholy timholy commented Oct 13, 2017

When building Julia, this keeps track of the files used by sysimg.jl, and then after building finishes it copies the contents of those files to a cache file <juliasrcdir>/usr/share/julia/base.cache. This allows Revise (and presumably other packages like Gallium) to detect changes that you've made to the source files that define Base. This makes it trivial to hack on Base without rebuilding Julia: you just say Revise.track(Base), and any previous and future edits to the source files get incorporated into your running session automatically.

In my opinion this change, in conjunction with timholy/Revise.jl#49, eliminates the last vestige of any kind of "dicey" behavior in Revise. (Previously, the need to parse the source to extract the list of included files was a major wart, because there were many ways that parsing could fail.) AFAIK its only limitations now stem from those of the fix to #265.

As a heads-up, once this merges I may start looking into what it will take to make Revise one of the "default packages" that gets installed & run by every user. (Configurably, of course.)

On Windows just a difference in backslash/forwardslash can cause a
matching failure, but we shouldn't care about such differences.
@timholy
Copy link
Member Author

timholy commented Oct 13, 2017

Hmm, while this works "on top of" an existing installation, in a fresh installation it fails because it seems I'm relying on stuff that hasn't finished installing/building. Specifically, the problem is the Makefile change that executes write_base_cache immediately after the *.so is compiled. Does anyone who knows more about build systems have any suggestions about how to structure this better? Of course I could "wire this in" to sysimg.jl, but somehow I thought it would be better as a separate post-compile step.

Makefile Outdated
@@ -226,7 +226,7 @@ $$(build_private_libdir)/sys$1.o: $$(build_private_libdir)/inference.ji $$(JULIA
mv [email protected] $$@; \
else \
echo '*** This error is usually fixed by running `make clean`. If the error persists$$(COMMA) try `make cleanall`. ***' && false; \
fi )
fi && $$(call spawn,$3) $$(JULIAHOME)/etc/write_base_cache.jl)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separate this out into it's own build step.

julia-base-cache: julia-sysimg-% | $(DIRS) $(build_datarootdir)/julia/base
    (call exec,$(JULIA_EXECUTABLE) $$JULIAHOME)/etc/write_base_cache.jl $(build_datarootdir)/julia/base/base.cache)

You might want to actually let make decide the location of the cache file and pass that to the script instead of letting the script discover if we are doing an out-of-source build, then you can add it to the list of dependencies for in L114 for julia-release julia-debug.

At least that's how I would start.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You saved me a bunch of time. Thanks so much.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though do you really think it should go in usr/share/julia/base? For me that's a symlink, so it actually ends up base/, which is why I ended up adding a .gitignore. What about just putting it in usr/share/julia directly?

Copy link
Member

@vchuravy vchuravy Oct 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That should be OK as well. If you want you can test out-of-tree builds and make install.
I am myself not a 100% sure where the right directory is :)

@timholy
Copy link
Member Author

timholy commented Oct 15, 2017

Tested out-of-tree builds and make install; both work (after I rebased on recent master due to #23997). Barring objections I'll merge this soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants