Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force recompilation if imported module has changed #3703

Merged
merged 8 commits into from
Sep 15, 2022

Conversation

hubertp
Copy link
Collaborator

@hubertp hubertp commented Sep 13, 2022

Pull Request Description

IR cache never really took into account a situation when a binding from the imported module has changed. In other words, it would continue to happily use the serialized metadata without noticing that it changed.

This change forces cache invalidation when any of the imported modules was invalidated (or rather not loaded from cache).

Important Notes

Added simple test infrastructure that simulates file modifications that would trigger the initial cache invalidation.
If they succeed, cache invalidation is propagated thus causing an error.

Checklist

Please include the following checklist in your PR:

  • The documentation has been updated if necessary.
  • All code conforms to the
    Scala,
    Java,
    and
    Rust
    style guides.
  • All code has been tested:
    • Unit tests have been written where possible.
    • If GUI codebase was changed: Enso GUI was tested when built using BOTH
      ./run ide build and ./run ide watch.

Copy link
Member

@radeusgd radeusgd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

As we don't have tests, we need to ensure manually that it works as expected:

  1. Did you try the scenario from the bug report to verify that now it behaves correctly?
  2. Can you somehow verify if after the changes there are not "too" many invalidations? (I guess just running a program twice, one with cleared caches and one cached would be ok to see that the second one is faster - very imperfect test but better than nothing)

@hubertp
Copy link
Collaborator Author

hubertp commented Sep 14, 2022

Looks good to me.

As we don't have tests, we need to ensure manually that it works as expected:

  1. Did you try the scenario from the bug report to verify that now it behaves correctly?
  2. Can you somehow verify if after the changes there are not "too" many invalidations? (I guess just running a program twice, one with cleared caches and one cached would be ok to see that the second one is faster - very imperfect test but better than nothing)

Yeah, I did all of that.
In the end I figured it wasn't so hard to setup tests for this. This was added in the last commit

Copy link
Member

@radeusgd radeusgd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's even better, great!

I really appreciate adding the tests, as not having tests really means we cannot guarantee we don't regress - see the REPL being broken so often.

@hubertp hubertp force-pushed the wip/hubert/cache-invalidation-182953733 branch from fead139 to 757eec9 Compare September 14, 2022 10:48
Copy link
Member

@JaroslavTulach JaroslavTulach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great test! One question about transitive dependencies & ordering. One question about including hashing with a chain. Otherwise, looks like a progress. Approving.

IR cache never really took into account a situation when a binding from
the imported module has changed. In other words, it would continue to
happily use the serialized metadata without noticing that it changed.

This change forces cache invalidation when any of the imported modules
was invalidated (or rather not loaded from cache).
Sadly, it doesn't appear that we have appropriate infrastructure to
write such scenarios so I'm delaying that bit for now.
Using _x, where x is the change step, to simulate modifications to files
and thus trigger cache invalidation that should propagate to upstream
modules.
@hubertp hubertp force-pushed the wip/hubert/cache-invalidation-182953733 branch from fed45c3 to 9293020 Compare September 15, 2022 10:07
@hubertp hubertp added the CI: Ready to merge This PR is eligible for automatic merge label Sep 15, 2022
@mergify mergify bot merged commit a044255 into develop Sep 15, 2022
@mergify mergify bot deleted the wip/hubert/cache-invalidation-182953733 branch September 15, 2022 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI: Ready to merge This PR is eligible for automatic merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants