JRuby support

[headius]

This issue describes a project to support JRuby in the debug library.

Background

JRuby has supported the same debugger as CRuby in the past via the ruby-debug gem (and to a lesser extent the old debug.rb). Back then, we worked with CRuby developers and IDE developers (such as those who worked on RubyMine from JetBrains) to have multiple backend libraries for ruby-debug that supported JRuby and CRuby.

In the intervening years, ruby-debug has bitrotted without any major updates in the past decade, and the new version of debug.rb has been greatly improved by @ko1 and others. This new debug.gem is now the standard for debugging Ruby code on CRuby.

ruby-debug today

The problem with ruby-debug today (other than it being very old and unmaintained) is that it depends on an older IRB that does not define any debug commands. As a result, when loading ruby-debug into a newer version of IRB (with debug.gem support), the commands that ruby-debug defines conflict, leading to errors and failing to boot.

$ rdebug
TypeError: superclass mismatch for class Continue
  <module:ExtendCommand> at /Users/headius/work/jruby/lib/ruby/gems/shared/gems/ruby-debug-0.11.0/cli/ruby-debug/commands/irb.rb:32
            <module:IRB> at /Users/headius/work/jruby/lib/ruby/gems/shared/gems/ruby-debug-0.11.0/cli/ruby-debug/commands/irb.rb:6
                  <main> at /Users/headius/work/jruby/lib/ruby/gems/shared/gems/ruby-debug-0.11.0/cli/ruby-debug/commands/irb.rb:5

The Continue class conflicts with a class of the same name added to IRB, and it's likely other classes would also conflict.

https://github.com/ruby-debug/ruby-debug/blob/ver_0_11_0/cli/ruby-debug/commands/irb.rb#L30-L33

It's likely that we can adapt ruby-debug for the IRB changes in the short term, but maintaining a completely separate debugging tool only for JRuby would not be efficient. So we desire to support debug.gem in JRuby.

Supporting JRuby

I have spent a little bit of time trying to understand the codebase and I think it is feasible (if difficult) for us to make debug.gem work with JRuby. Several things in JRuby are different, however, and would require some adaptation:

• There's a C extension mostly to access runtime call structures (frames etc) and instruction sequences (iseq).

We might need an extension in JRuby, or we could possibly just access these structures from Ruby code using our Java integration (but that Ruby code would have to be exempt from debug hooks, somehow).

• Breakpoints are installed at an instruction sequence level.

We would need to be able to support this in our instruction set.

• JRuby's instruction sequences are very different from CRuby's (some of the time).

JRuby has three different instruction sets: our "startup" set, which is a simple instruction set designed to parse, compile, and start executing quickly; our "full" instruction set, produced after optimization passes and providing a full control flow and data flow graph structure; and JVM bytecode, which we lazily compile to once code is hot.

The "startup" instruction set is the obvious target for debug.gem support. It is the simplest structure and would map most naturally to CRuby. It does not match CRuby exactly, however, so we would ned to adapt debug.gem to support both CRuby and JRuby instruction sets.

• JRuby would support debug.gem only in our "startup" mode with the simplest instructions and no optimization.

This should be sufficient for most debugging use cases, but it would obviously be slower than running one of our our optimized modes.

Alternatives

• We could just update and maintain ruby-debug. However, it's a large codebase and we are very limited on developer resources.
• We could provide better support for JVM debugging. JRuby can be set to always compile Ruby code directly into JVM bytecode, and in this mode most Ruby methods would work properly with JVM debugging APIs. However, the structure of classes and objects in JRuby is quite different from standard JVM objects, so setting watches and exploring the heap would be rather foreign.
• We could again try to work with IDE vendors like JetBrains to support debugging. Unfortunately JetBrains has reduced efforts to support Ruby in recent years and the last time I reached out to them they were not interested in major RubyMine work.