Add support for TruffleRuby in TravisCI

[eregon]

Hello,

I would like to have support for TruffleRuby in TravisCI.
So that TravisCI users can use TruffleRuby with, in their .travis.yml:

matrix:
  include:
  - rvm: 2.5.1
  - rvm: truffleruby

What is required to make this work?
This is not very clear to me.

I would like to let TravisCI users try TruffleRuby releases. Therefore it is not a goal to have "truffleruby-head" version for now.

TruffleRuby can now be installed by RVM and is part of RVM master.
So I guess one requirement is waiting for a stable release of RVM as TravisCI seems to not use the latest RVM. Probably TravisCI uses the latest stable release of RVM? Could you confirm?

I tried to manually update RVM with rvm get head, and then install TruffleRuby with the command that TravisCI seems to use for rvm: (eregon/mspec@a7221c5 for details). However, this fails:

$ rvm use truffleruby --install --binary --fuzzy
curl: (22) The requested URL returned error: 403 Forbidden
curl: (22) The requested URL returned error: 404 Not Found
Required truffleruby is not installed - installing.
curl: (22) The requested URL returned error: 403 Forbidden
curl: (22) The requested URL returned error: 404 Not Found
Searching for binary rubies, this might take some time.
Requested binary installation but no rubies are available to download, consider skipping --binary flag.
Gemset '' does not exist, 'rvm truffleruby do rvm gemset create ' first, or append '--create'.

I then tried without the --binary flag and that worked (with sudo: true to let RVM install dependencies).
That was rather slow, due to compilation and installing the default gems.

What should be done to make rvm: truffleruby work?
Should we create a "binary ruby" of TruffleRuby?
Should we have one archive per OS, distribution & version?
Where should that be hosted?

Currently, TruffleRuby is distributed as a single archive per OS (Linux or macOS):
https://github.com/oracle/truffleruby/releases/download/vm-1.0.0-rc2/truffleruby-1.0.0-rc2-linux-amd64.tar.gz
https://github.com/oracle/truffleruby/releases/download/vm-1.0.0-rc2/truffleruby-1.0.0-rc2-macos-amd64.tar.gz

The only step that is more fined-grained than Linux/macOS is recompiling the OpenSSL C extension, as that depends on the version and configuration of the system libssl.
By default, the OpenSSL C extension is compiled against libssl 1.0.2, which might work out of the box on some of the Travis images.
What versions of libssl does TravisCI images use?

cc @BanzaiMan @rkh @mpapis

[eregon]

However, it seems the wrong Bundler or gem environment was used:
https://travis-ci.org/eregon/mspec/jobs/394187619

That was due to TruffleRuby not supporting $RUBYLIB yet, which bundle exec relies on when bundle install --deployment is used.
In any case, unrelated to TravisCI, except that --deployment is used by default.

[fniephaus]

Starting with ruby.rb, it looks like you might want to have a look at rvm.rb to understand how rvm is invoked. It seems like TruffleRuby needs to be supported by rvm first and then it might work without further ado? Not sure if someone is already working on this.
With regard to compatibility to Travis build environments, I recommend setting up a private repository with Travis Pro which allows you to start a debugging sessions which in turn gives you ssh access to a real Travis worker. I use this quite often for smalltalkCI which is also integrated with Travis.

Hope this helps,
Fabio

[eregon]

@fniephaus Thank you for the pointers!
TruffleRuby is already supported by RVM, when building from source: rvm/rvm#4406
The question is: what do we need next, beyond a stable release of RVM?

It looks like we need a "binary ruby" of TruffleRuby due to Travis doing rvm use #{ruby_version} --install --binary --fuzzy (from rvm.rb) or we need to modify rvm.rb (for instance to drop the --binary flag for TruffleRuby), which might be simpler actually.

[fniephaus]

I assume --binary means some sort of bottled installation of precompiled binaries? How long does it take to build TruffleRuby with rmv? You might want to save that time during builds if it takes much longer than downloading some binary blob.
Also, why not make a TruffleRuby binary available via SubstrateVM?
In terms of hosting, I would go with GitHub releases because Travis already optimizes network connectivity for GitHub :)

[eregon]

@fniephaus It's RVM's concept of "ruby binary".
We already have binary releases, on GitHub, as shown above.
One tricky bit though is the OpenSSL version might change from system to system and if it's not 1.0.2, we need to recompile the Ruby OpenSSL C extension.
So one task is actually to check what OpenSSL versions the various Travis images use.

[eregon]

How long does it take to build TruffleRuby with rvm?

Not that long, < 1 minute locally. But it's slower on Travis CI and RVM does a lot of extra things beyond the download for rvm install (or rvm use --install), notably installing system packages, which takes around 4min.

Downloading the archive manually seems quite a bit faster (only a few seconds for download+extract), and recompiling the OpenSSL extension seems to take a bit more than a minute: https://travis-ci.org/ruby/mspec/jobs/397422999

[eregon]

So one task is actually to check what OpenSSL versions the various Travis images use.

I checked and it seems all 4 modes documented by Travis CI at https://docs.travis-ci.com/user/reference/overview/#Virtualisation-Environment-vs-Operating-System use OpenSSL 1.0.1 or 1.0.2:

$ ruby -ropenssl -e 'p OpenSSL::OPENSSL_LIBRARY_VERSION'
precise:          "OpenSSL 1.0.1 14 Mar 2012"
trusty with sudo: "OpenSSL 1.0.1f 6 Jan 2014"
trusty container: "OpenSSL 1.0.1f 6 Jan 2014"
macOS:            "OpenSSL 1.0.2l  25 May 2017"

I think the OpenSSL extension as shipped in TruffleRuby 1.0.0-rc3 is compatible with all of these, but I need to check.

[fniephaus]

4min is quite some time if you want people to use TruffleRuby for short test cases or scripts.
Probably best to set up some infrastructure to generate TruffleRuby binaries with OpenSSL extensions that are compatible with Travis.

[eregon]

With TruffleRuby 1.0.0-rc3, it seems we don't need to recompile the OpenSSL extension on Travis CI: I ran successfully the OpenSSL specs & MRI tests on the 4 modes mentioned above.

So it's now a matter of integrating with RVM support in Travis-build's rvm.rb, or integrating with RVM's "binary rubies".
The binaries at https://github.com/oracle/truffleruby/releases seem to have everything needed, except bundler which is not part of the archive.

[rkh]

It might be possible to build and store binaries via https://github.com/travis-ci/travis-rubies, it's what we do for other Ruby implementations.

[eregon]

Note to anyone reading this: you can already run TruffleRuby in TravisCI, see https://github.com/oracle/truffleruby/blob/master/doc/user/standalone-distribution.md#testing-truffleruby-in-travisci

We're working on getting support for rvm install --binary truffleruby, but I guess it will be a while until TravisCI updates to a more recent RVM.
There should be no need for an external download location beyond GitHub downloads for TruffleRuby, except if we want to optimize by e.g., pre-installing bundler (which is not done for any other Ruby currently).

[ioquatix]

I tried on travis:

rvm get head
rvm install truffleruby

But it failed: https://travis-ci.org/socketry/timers/jobs/451176353

[eregon]

@ioquatix Thanks for the report. It seems to fail in RVM's code installing dependencies, I'll take a look.

[ioquatix]

I would like to make it work. But it has to be super simple. I don't mind adding rvm get head at the very top of the travis config for all builds.

[eregon]

I fixed this issue with travis-ci/travis-build#1604

Now

rvm:
  - truffleruby

should just work^TM, and is documented in https://docs.travis-ci.com/user/languages/ruby/#truffleruby !

[eregon]

Note, on Xenial the OpenSSL version is:

$ ruby -ropenssl -e 'p OpenSSL::OPENSSL_LIBRARY_VERSION'
precise:          "OpenSSL 1.0.1 14 Mar 2012"
trusty with sudo: "OpenSSL 1.0.1f 6 Jan 2014"
trusty container: "OpenSSL 1.0.1f 6 Jan 2014"
macOS:            "OpenSSL 1.0.2l  25 May 2017"
xenial:           "OpenSSL 1.0.2g  1 Mar 2016"

This could be related to some OpenSSL-related errors we get recently with TruffleRuby in TravisCI.