Metaissue: Improve the interpreter

[straight-shoota]

What's missing for the interpreter:

• crystal i, a Crystal interpreter #11159
• Type promotion for variadic arguments (Interpreter: Promote arguments of variadic function calls #11795)
• Crystal::Loader integration (Use Crystal::Loader in the interpreter #11579)
• Syscalls
• Pending lib bindings (libxml, libyaml)
  • libyml (Enable interpreter integration specs for YAML #11801)
  • libxml2 (Segfault from FFI closure function in interpreter #11802)
• Musl libc support
  • Support for statically linked compiler
• Check concurrency implementation
• Documentation (manpages, crystal-book)
• Improved REPL UX
  • Specs for REPL (Fix for Crystal interpreter crash #11717 (comment))
  • History and line edit (Add readline for the compiler #11340)
  • Syntax highlighting (Refactor syntax highlighter and add ANSI escape code highlighter for console #11366)
• Windows support (Enable the interpreter on Windows #14964)
  • Runtime loader for windows (Implement Crystal::Loader for MSVC #12140)
• asm support (interpreter bytecode)
• Stack alignment (LLVM 18 breaks 128-bit integers in the interpreter #14832)
• Arm support ( Support interpreter on arm #12299)
• More specs
• 100 other TODOs in src/compiler/crystal/repl

Feel free to post comments for additional items here.

No discussion on individual items. Thank you 🙇‍♂️

That should be moved to dedicated issues. If none exists, please open one or start an informal discussion at https://forum.crystal-lang.org/

[asterite]

I think pry doesn't work well in many cases. For example, block arguments are not visible. In general, I feel that pry has still a long way to go

[asterite]

After #11578 is merged, the first thing I would do is to write at least one spec for debugger and pry. There's a lot to fix and improve there, and without tests it's going to be tough.

My idea was to have tests specify three things:

• a source, which has a debugger call in it
• given input
• expected output

For example:

assert_interpreter_debugger(
  code: "a = 1; debugger; a",
  input: "a",
  expected_output: "1",
)

That is, after debugger is reached, the user types "a" on the console, and they expect to get "1" as the output.

This will require adapting the interpreter so that the input and output are not tied to STDIN and STDERR, which might also eventually make it possible to hook the interpreter into code editors.

These tests will also be independent of the underlying implementation, so they will be extremely useful and resilient to changes.

With those tests in place, I could start working on some of them to fix existing bugs, like #11555 (comment)

[beta-ziliani]

@asterite just to clarify, each of the lines stating BUG: missing handling... is a TODO waiting to be completed, right? (Just stumble across one of this).

[asterite]

That's it! Maybe I should have put a TODO there 😅

[watzon]

I assume the issue that happens when attempting to require "openssl" or anything that relies on openssl is included in one of these subtasks? The issue seems to be different based on whether you require openssl in the repl or in a file the interpreter is running, but both break.

If other people aren't having this problem for some reason I'll gladly make a new issue.

[HertzDevil]

That is, after debugger is reached, the user types "a" on the console, and they expect to get "1" as the output.

#12119 does this for LLDB. We could probably use FileCheck here or even share the same test source files for multiple CLI debuggers (another one I have in my mind is GDB).

[mwgkgk]

https://github.com/Ivo-Balbaert/programming_crystal/blob/master/exercises/crystal_new/fibers.cr

Something really slow about channels. crystal i <filename> this thing grinds my computer to a halt i barely Ctrl-c'd at 1507 out of 10000 iterations. crystal r <filename> completes smoothly in a blink of an eye. Linux x86-64

# START:p1
chan = Channel(String).new
i = 0
num = 10000
num.times do
  spawn do
    chan.send "fiber #{i}: I like crystals!"
  end
  i += 1
  puts chan.receive
end
# =>
# fiber 1: I like crystals!
# fiber 2: I like crystals!
# fiber 3: I like crystals!
# fiber 4: I like crystals!
# ...
# fiber 10000: I like crystals!
# END:p1

[mdwagner]

Can confirm similar results on MacOS (Intel) with Crystal 1.5.0

[asterite]

Yes, the interpreter is slower than compiled mode. And I have no immediate plans to improve its speed.