Bundler 1.10.6 can't find gems with invalid gemspec files

[GUI]

I have a custom gem that is installed from a git source in my Gemfile. After upgrading from Bundler 1.9 to Bundler 1.10.6, I started receiving errors that the gem could not be found when trying to load anything that interacted with the gem (in this example, it's spinning up a rails server):

$ rails s
Could not find mygem-0.1.0 in any of the sources
Run `bundle install` to install missing gems.

After downgrading to 1.10.5, things would still fail to start, but I at least got a more helpful error that led me to the actual problem:

$ rails s
The gemspec at /Users/nmuerdte/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bundler/gems/mygem-51004a1371dd/mygem.gemspec is not valid. The validation error was '"FIXME" or "TODO" is not a description'

So while this is certainly a solvable problem with my gem (since the gemspec is indeed invalid), the current behavior in 1.10.6 is somewhat confusing, since bundle install reports installing the gem without any warnings, but then when trying to load the gem it simply reports it as missing.

I think the change in behavior is related to this change: #3856 I think the attempt with that pull request was to actually allow these invalid gemspecs to continue working on 1.10, but it seems like it's perhaps just deferred the error to later when the loading fails (but in a less obvious way). Did something else maybe change earlier on the 1.10 branch that's completely excluding these invalid gemspecs from the gems Bundler will load?

I'm seeing this under Ruby 2.2.1 and RubyGems 2.4.5.

Thanks!

[segiddins]

This was necessary to ensure backwards compatibility, but we'll go back to rejecting invalid gemspecs entirely in 2.0.

[indirect]

Which gemspecs loaded on 1.9 but not on 1.10? Gems that have stopped working are a bug, and we should fix it.

[GUI]

Sorry for not providing a more concrete example upfront. The default gemspec files generated by bundler actually exhibit this problem (since they have TODOs in the descriptions, which rubygems doesn't like--this is actually what bit us on several internal gems where we had been lazy and never bothered to fill out the descriptions in the gemspec).

Here's an example gem with a gemspec that demonstrates this problem: https://github.com/GUI/test_bundler_invalid_gem This gem was generated by running bundle gem test_bundler_invalid_gem and committing the results.

Then for a simple demonstration of the issue, here's a minimal Gemfile:

source "https://rubygems.org"
gem "test_bundler_invalid_gem", :git => "https://github.com/GUI/test_bundler_invalid_gem.git"

Under Bundler 1.9.10, this dependency works and exhibits no warnings:

$ bundle --version
Bundler version 1.9.10

$ bundle
Updating https://github.com/GUI/test_bundler_invalid_gem.git
Fetching gem metadata from https://rubygems.org/.........
Fetching version metadata from https://rubygems.org/..
Resolving dependencies...
Using test_bundler_invalid_gem 0.1.0 from https://github.com/GUI/test_bundler_invalid_gem.git (at master)
Using bundler 1.9.10
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.

$ bundle exec irb
irb(main):001:0> 

Under Bundler 1.10.5, this dependency causes bundle exec commands to fail and exit, but generates an error message pointing to the culprit:

$ bundle --version
Bundler version 1.10.5

$ bundle
Updating https://github.com/GUI/test_bundler_invalid_gem.git
Fetching gem metadata from https://rubygems.org/.........
Fetching version metadata from https://rubygems.org/..
Resolving dependencies...
Using test_bundler_invalid_gem 0.1.0 from https://github.com/GUI/test_bundler_invalid_gem.git (at master)
Using bundler 1.10.5
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.

$ bundle exec irb
The gemspec at /Users/nmuerdte/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bundler/gems/test_bundler_invalid_gem-f937df01aeab/test_bundler_invalid_gem.gemspec is not valid. The validation error was '"FIXME" or "TODO" is not a description'

Under 1.10.6, bundle exec commands similarly fail and exit like in 1.10.5. However, the behavior here is a bit more confusing, since it simply says the gem isn't installed and to run bundle install again (which does report installing the gem successfully and has no effect).

$ bundle --version
Bundler version 1.10.6

$ bundle
Updating https://github.com/GUI/test_bundler_invalid_gem.git
Fetching gem metadata from https://rubygems.org/.........
Fetching version metadata from https://rubygems.org/..
Resolving dependencies...
Using test_bundler_invalid_gem 0.1.0 from https://github.com/GUI/test_bundler_invalid_gem.git (at master)
Using bundler 1.10.6
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.

$ bundle exec irb
Could not find test_bundler_invalid_gem-0.1.0 in any of the sources
Run `bundle install` to install missing gems.

Obviously, all of this can be resolved by removing the TODOs in the gemspecs and having valid gemspec files. But these are gemspecs that did previously work under 1.9, and 1.10.6's lack of straightforward error message (claiming the gem doesn't exist), was what particularly caused some confusion for us.

Thanks!

[segiddins]

Ah, so that is a bug!

[agis]

Thing is, at this point Bundler.ui is the silent one (UI::Silent), so the warning is never printed.

An easy fix is to do instead UI::Shell.new.warn, or even set the ui at that point (ie. Bundler.ui = UI::Shell.new). However I'm not sure it's the correct approach.

@segiddins WDYT?

[indirect]

Ugh, that's right. Maybe we need to revert the silencing changes as well? :(

[segiddins]

@indirect for now, let's just go with UI::Shell.new.warn ?

[indirect]

Works for me.

[onemanstartup]

The gemspec at /Users/onemanstartup/.gem/ruby/2.3.0/bundler/gems/wiselinks-ba85c7202d2e/wiselinks.gemspec is not valid. The validation error was 'duplicate dependency on rspec (>= 0, development), (>= 0) use:
    add_development_dependency 'rspec', '>= 0', '>= 0'
'
Could not find wiselinks-1.2.2 in any of the sources
Run `bundle install` to install missing gems.

I'm using wiselinks from this branch https://github.com/aishek/wiselinks/tree/ie8-fallback-fix

❯ bundler -v                                                                                                    2.3.0p0
Bundler version 1.11.2

So is this error is still error?

[indirect]

@onemanstartup yes, the gemspec is still invalid--please remove the duplicate dependency so that rubygems can validate the gem.

[onemanstartup]

@indirect it works. strange thing, now when I install fixed version i can install now previous bugged version. Anyway, thanks.

[SurajShukla]

I am not sure If this can be put here or not but maybe someone can figure out.

I am getting this gemspec related issue too.

The gemspec at
/home/users/Projects/new_project/vendor/gems/spree_social/spree_social.gemspec
is not valid. Please fix this gemspec.
The validation error was 'spree_social-3.1.0.beta contains itself
(spree_social-3.1.0.beta.gem), check your files list'

I am not able to figure out the solution.