Skip to content

Commit

Permalink
ci: workaround recent bundler release bug
Browse files Browse the repository at this point in the history 
I *think* this is related to:

- rubygems/rubygems#5871
- (proposed fix) rubygems/rubygems#5875
  • Loading branch information
@flavorjones
flavorjones committed Aug 27, 2022
1 parent 839f079 commit b2adf01
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion scripts/test-gem-install
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ if [ -n "${BUNDLE_APP_CONFIG:-}" ] ; then
export BUNDLE_CACHE_PATH="${BUNDLE_APP_CONFIG}/cache"
fi

gem install bundler -v "~> 2.2"
gem install bundler -v "~> 2.2, != 2.3.21"
bundle install --local || bundle install

rm -rf lib ext # ensure we don't use the local files
Expand Down

3 comments on commit b2adf01

@Nakilon
Copy link

@Nakilon Nakilon commented on b2adf01 Dec 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this slowness on alpine for already like three years. Any host OS (macOS, Ubuntu, RHEL), different versions of Ruby, any cloud provider.
Finally got to the lldb and found it's doing some versions parsing billions of times (for example, the last method on stack was canonical_segments, the it's returning value below) and it's doing it for just a single command gem install -N bundler:2.4.22 that was itself supposed to fix the issue if there is any in the gem that is in default docker image. But it takes eternity. Why does it parse anything at all if it's just a single gem install command with a specified version? It's doing it right after downloading the specs.gz.

It is so weird I have this issue every time, for years. It was a pain for hobby projects but now it's on the way in my job.

...
[13, 12, "pre", "alpha", "PLAY", 1051, "removinghighchartsdependency", 1551]
[13, 12, "pre", "alpha", "PLAY", 880, "cardkithighlightzindex", 1655]
[13, 12, "pre", "alpha", "PLAY", 1081, "exportingtypes", 1627]
[13, 12, "pre", "alpha", "PLAY", 1051, "removinghighchartsdependency", 1551]
[13, 12, "pre", "alpha", "PLAY", 1081, "exportingtypes", 1627]
[13, 12, "pre", "alpha", "PLAY", 1051, "removinghighchartsdependency", 1551]
[13, 12, "pre", "alpha", "PLAY", 1081, "exportingtypes", 1627]
[13, 12, "pre", "alpha", "PLAY", 1081, "exportingtypes", 1626]
[13, 12, "pre", "alpha", "PLAY", 1081, "exportingtypes", 1627]
[13, 12, "pre", "alpha", "PLAY", 1081, "exportingtypes", 1626]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1611]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1608]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1611]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1608]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1611]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1609]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1611]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1609]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1623]
[13, 12, "pre", "alpha", "PLAY", 1081, "exportingtypes", 1627]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1623]
[13, 12, "pre", "alpha", "PLAY", 1081, "exportingtypes", 1627]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1623]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1611]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1623]
[13, 12, "pre", "alpha", "PLAY", 1081, "exporttypes", 1611]
^Z[1]+  Stopped                    time gem install -N bundler:2.4.22
/ #

Looks like it parses the versions of all the gems on rubygems, why?...

And I write here , because it's

bundler (default: 2.1.4)

for me, not the one in the commit.

@Nakilon
Copy link

@Nakilon Nakilon commented on b2adf01 Dec 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example backtrace:

	from /usr/local/bin/gem:21:in `<main>'
	from /usr/local/lib/ruby/2.7.0/rubygems/gem_runner.rb:59:in `run'
	from /usr/local/lib/ruby/2.7.0/rubygems/command_manager.rb:148:in `run'
	from /usr/local/lib/ruby/2.7.0/rubygems/command_manager.rb:178:in `process_args'
	from /usr/local/lib/ruby/2.7.0/rubygems/command.rb:325:in `invoke_with_build_args'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:165:in `execute'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:217:in `install_gems'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:217:in `each'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:224:in `block in install_gems'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:199:in `install_gem'
	from /usr/local/lib/ruby/2.7.0/rubygems/dependency_installer.rb:394:in `resolve_dependencies'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/installer_set.rb:56:in `add_always_install'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/installer_set.rb:155:in `find_all'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/best_set.rb:29:in `find_all'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/best_set.rb:23:in `pick_sets'
	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each_source'
	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/best_set.rb:24:in `block in pick_sets'
	from /usr/local/lib/ruby/2.7.0/rubygems/source.rb:86:in `dependency_resolver_set'
	from /usr/local/lib/ruby/2.7.0/rubygems/source.rb:90:in `rescue in dependency_resolver_set'
	from /usr/local/lib/ruby/2.7.0/rubygems/source.rb:90:in `new'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/index_set.rb:22:in `initialize'
	from /usr/local/lib/ruby/2.7.0/rubygems/spec_fetcher.rb:214:in `available_specs'
	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each_source'
	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each'
	from /usr/local/lib/ruby/2.7.0/rubygems/spec_fetcher.rb:226:in `block in available_specs'
	from /usr/local/lib/ruby/2.7.0/rubygems/spec_fetcher.rb:226:in `sort'
	from /usr/local/lib/ruby/2.7.0/rubygems/name_tuple.rb:95:in `<=>'
	from /usr/local/lib/ruby/2.7.0/rubygems/name_tuple.rb:95:in `<=>'
	from /usr/local/lib/ruby/2.7.0/rubygems/version.rb:347:in `<=>'
	from /usr/local/lib/ruby/2.7.0/rubygems/version.rb:378:in `canonical_segments'

UPD:
even with flag --ignore-dependencies it's still doing it:

	from /usr/local/bin/gem:21:in `<main>'
	from /usr/local/lib/ruby/2.7.0/rubygems/gem_runner.rb:59:in `run'
	from /usr/local/lib/ruby/2.7.0/rubygems/command_manager.rb:148:in `run'
	from /usr/local/lib/ruby/2.7.0/rubygems/command_manager.rb:178:in `process_args'
	from /usr/local/lib/ruby/2.7.0/rubygems/command.rb:325:in `invoke_with_build_args'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:165:in `execute'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:217:in `install_gems'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:217:in `each'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:224:in `block in install_gems'
	from /usr/local/lib/ruby/2.7.0/rubygems/commands/install_command.rb:199:in `install_gem'
	from /usr/local/lib/ruby/2.7.0/rubygems/dependency_installer.rb:394:in `resolve_dependencies'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/installer_set.rb:56:in `add_always_install'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/installer_set.rb:155:in `find_all'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/best_set.rb:29:in `find_all'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/best_set.rb:23:in `pick_sets'
	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each_source'
	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/best_set.rb:24:in `block in pick_sets'
	from /usr/local/lib/ruby/2.7.0/rubygems/source.rb:86:in `dependency_resolver_set'
	from /usr/local/lib/ruby/2.7.0/rubygems/source.rb:90:in `rescue in dependency_resolver_set'
	from /usr/local/lib/ruby/2.7.0/rubygems/source.rb:90:in `new'
	from /usr/local/lib/ruby/2.7.0/rubygems/resolver/index_set.rb:22:in `initialize'
	from /usr/local/lib/ruby/2.7.0/rubygems/spec_fetcher.rb:214:in `available_specs'
	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each_source'
	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each'
	from /usr/local/lib/ruby/2.7.0/rubygems/spec_fetcher.rb:226:in `block in available_specs'
	from /usr/local/lib/ruby/2.7.0/rubygems/spec_fetcher.rb:226:in `sort'
	from /usr/local/lib/ruby/2.7.0/rubygems/name_tuple.rb:95:in `<=>'
	from /usr/local/lib/ruby/2.7.0/rubygems/name_tuple.rb:95:in `<=>'
	from /usr/local/lib/ruby/2.7.0/rubygems/version.rb:349:in `<=>'
	from /usr/local/lib/ruby/2.7.0/rubygems/version.rb:378:in `canonical_segments'

I see the double nested this:

	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each_source'
	from /usr/local/lib/ruby/2.7.0/rubygems/source_list.rb:100:in `each'

UPD2:
as I understand it goes to add_always before even looking at the install flag, and goes on with some dependencies resolving. Which is for some reason slow.

UPD3: looks like with the flag --ignore-dependencies the attribute @ignore_dependencies in find_all is false while the value is true during the options parsing. Is it being later overriden somewhere?

@flavorjones
Copy link
Member Author

@flavorjones flavorjones commented on b2adf01 Dec 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nakilon Why are you commenting on a two-year old commit in an unrelated code base with information about an issue that should be filed upstream against rubygems?

Please sign in to comment.