Use single cache file per library

[JaroslavTulach]

Follow up of

• Store IR caches in suitable format to read them "on demand" #8207
• Investigate why certain .ir files are bigger in the new format #8323

One of the ideas for future work in #8207 is to benefit from huge cache files mmapped into memory. Rather than having a lot of small .ir files for each module in standard library, let's have just a single file per each library. Btw. there already is such a file for BindingsMaps per each library - with the new format we should include whole IR in there (as its parts are loadable lazily on demand)

Tasks

Preview Give feedback

1. Split SerializationManager into independent parts

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-01-30):

Progress: - fixing GraalVM 24+ "upgrade": 88fd039

• successfully using the proposed onDeniedThreadAccess API in Enso: Allow control of throwDeniedThreadAccess via TruffleContext.threadAccessDeniedHandler oracle/graal#8266 (comment)
• licenses: Upgrading to GraalVM 21.0.2 #8883 (comment)
• Simplifying runtime-compiler dependencies: Towards simplifying runtime-compiler dependencies #8894
• reviewing Akka test fixes: Reduced resources for various Akka dispatchers #8892 (review)
• review of line by line parser: Add parser for line by line processing #8719 (comment)
• planning It should be finished by 2024-02-05.

Next Day: Dependencies & rewriting caching

[enso-bot]

Jaroslav Tulach reports a new STANDUP for today (2024-01-31):

Progress: - Maven download tutorial: #8912

• always run with -ea PR: Always runEngineDistribution with -ea #8915
• standups, meetings It should be finished by 2024-02-05.

Next Day: Dependencies & rewriting caching

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-02-01):

Progress: - single cache per library PR: #8924

• Merged in -ea by default: Always runEngineDistribution with -ea #8915
• Executing .enso files in IGV and VSCode: Execute and debug .enso files with bin/enso in VSCode #8923
• IGV AU bugfix: [GR-52025] Use update center for NetBeans 14 in IGV oracle/graal#8290 It should be finished by 2024-02-05.

Next Day: Rewriting caching & VSCode extension

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-02-02):

Progress: - Better Enso testing: #8940

• Failing test report: Failing test runEngineDistribution --run test/Base_Tests Instrument on subsequent execution #8941
• got ./bin/enso debugging working: Execute and debug .enso files with bin/enso in VSCode #8923 (comment)
• review of Nothing: Fix Truffle invariants for Nothing with a warning #8764 (review)
• The Cache intention: https://discord.com/channels/@me/959434339182067712/1202947911864811521
• removal of Cache.Metadata: 22ac510
• fighting with SerializationManager
• standups It should be finished by 2024-02-05.

Next Day: Rewriting caching & VSCode extension debugging

Discord

Discord - A New Way to Chat with Friends & Communities

Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-02-03):

Progress: - introducing SerializationPool: 300972d

• Analyzing benchmarks: Store whole IR.Module in .bindings cache #8924 (comment)
• Removing SerializationManager: 4b75e45 It should be finished by 2024-02-05.

Next Day: Polishing cache rewrite for integration & VSCode extension debugging

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-02-04):

Progress: - fixing unit tests: d663e06

• Thread.join to make sure threads are closed
• Fixing SerializationManagerTest: 5a99f95
• crying over obstacles: https://discord.com/channels/@me/948519891961544714/1203619100920782949
• fixing runtime-fat-jar: Better runEngineDistribution --run test/Base_Tests "filter" #8940 (comment) It should be finished by 2024-02-05.

Next Day: Polishing cache rewrite for integration & VSCode extension debugging

Discord

Discord - A New Way to Chat with Friends & Communities

Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-02-05):

Progress: - CI fighting: https://github.com/enso-org/enso/actions/runs/7773475572/job/21211975666?pr=8924

• one more build.sbt bugfix: Keep only module-info.class in the root of runtime-fat-jar #8961
• discussing heap dump
• reply to GraalVM guys: Allow control of throwDeniedThreadAccess via TruffleContext.threadAccessDeniedHandler oracle/graal#8266 (comment)
• discussing Graal.js websocket It should be finished by 2024-02-05.

Next Day: Address Cache review comments & VSCode extension debugging

GitHub

Store whole IR.Module in .bindings cache · enso-org/enso@77b1369

Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.

[enso-bot]

Jaroslav Tulach reports a new 🔴 DELAY for yesterday (2024-02-05):

Summary: There is 7 days delay in implementation of the Use single cache file per library (#8639) task.
It will cause 7 days delay for the delivery of this weekly plan.

I am working at most at 30% capacity this week.

Delay Cause: The issue is in review, but it will need some time to finish the reviews and address the comments.

Possible solutions: I could rush with integration and then create follow up tickets, but given the reviewing only started, it seems better to give it some time and admit delay.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-02-06):

Progress: - reacting to review & announcing delay: #8924

• face to face review with Hubert
• fixed StackOverflowError: 947824e
• planning meeting It should be finished by 2024-02-12.

Next Day: Address Cache review comments & VSCode extension debugging

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-02-07):

Progress: - mmap only larger files: c061494

• fighting with swallowed exception: Store whole IR.Module in .bindings cache #8924 (comment)
• reviews: Local project manager work against any folder #8985 (review)
• testing framework discussions
• Big_Decimal discussions It should be finished by 2024-02-12.

Next Day: Address Cache review comments & VSCode extension debugging

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-02-08):

Progress: - addressing review comments: 5a4bfa9

• fixing NPE: a2b2945#r1482382954
• improving debugging in VSCode: 16518d6 It should be finished by 2024-02-12.

Next Day: Integrate .bindings cache & VSCode extension debugging

[enso-bot]

Jaroslav Tulach reports a new STANDUP for today (2024-02-09):

Progress: - Single .bindings cache integrated: #8924

• documentation for VSCode: d87ba6e
• VSCode extension ready for release: Execute and debug .enso files with bin/enso in VSCode #8923 (comment) It should be finished by 2024-02-12.

Next Day: Write tests for Graal.js multithreding issues