Cached catalogs + hiera-puppet-helper

[barrkel-m20]

I noticed, after several hours of debugging, that catalogs appear to be cached in a class variable and end up being reused across multiple Rspec examples even though parameterization data (let(:hiera_data) {} via https://github.com/mthibaut/hiera-puppet-helper) has changed.

This line does the caching:

rspec-puppet/lib/rspec-puppet/support.rb

Line 162 in 389f99e

@@cache[args] ||= self.build_catalog_without_cache(*args)

Why is this? It doesn't appear to be responsible for a major speedup, as far as I can tell, yet has major potential for test pollution.

The symptom is that later run examples get hiera data from earlier examples.

[ngiger]

Thanks for pointing to this issue.
Adding in my spec_helper.rb

module RSpec::Puppet
  module Support
    def build_catalog(*args)
      @@cache[args] = self.build_catalog_without_cache(*args)
    end
  end
end

avoided reusing hiera variables from a separate file.

[ngiger]

Hi

My spec_helper looks a little bit different

require 'rspec-puppet'
require 'puppetlabs_spec_helper/module_spec_helper'

fixture_path = File.expand_path(File.join(FILE, '..', 'fixtures'))

TODO: Hope this bugs gets squashed wit a new version of rspec

needed if bundle exec rspec spec/classes/ fails, but each spec/*.spec is

okay when run alone # see #215
module RSpec::Puppet

module Support

def build_catalog(*args)

  @@cache[args] = self.build_catalog_without_cache(*args)

end

end

end if false

RSpec.configure do |c|

c.module_path = File.join(fixture_path, 'modules')
c.manifest_dir = File.join(fixture_path, 'manifests')

end
at_exit { RSpec::Puppet::Coverage.report! }

I used this work-around in the following repositories
https://github.com/ngiger/puppet-desktop
https://github.com/ngiger/puppet-x2go
https://github.com/ngiger/puppet-elexis
in case you want to see it in action.

best regards

Niklaus

Am Montag, 17. November 2014, 12.32:44 schrieb Guyllaume Cardinal:

Stumbled upon the same problem as barrkel-m20 and tried to implement your
solution, but it didn't work. I'm guessing I just didn't implement it
properly. Could you tell me if my spec_helper.rb file looks ok?

require 'rspec-puppet'
require 'hiera-puppet-helper'

fixture_path = File.expand_path(File.join(__FILE__, '..', 'integration',

'fixtures'))

module RSpec::Puppet
  module Support
    def build_catalog(*args)
      @@cache[args] = self.build_catalog_without_cache(*args)
    end
  end
end

RSpec.configure do |c|
  c.module_path = File.join(fixture_path, 'modules')
  c.manifest_dir = File.join(fixture_path, 'manifests')
end

---

Reply to this email directly or view it on GitHub:
#215 (comment)

[gCardinal]

In the end I managed to make it work (which is why I deleted my question), but thank you for the code sample. I really appreciate you taking the time.

Here's mine, in the event someone comes across this issue:

require 'rspec-puppet'
require 'hiera-puppet-helper'

fixture_path = File.expand_path(File.join(__FILE__, '..', 'integration', 'fixtures'))

module RSpec::Puppet
  module Support
    def build_catalog(*args)
      @@cache[args] = self.build_catalog_without_cache(*args)
    end
  end
end

RSpec.configure do |c|
  c.module_path = File.join(fixture_path, 'modules')
  c.manifest_dir = File.join(fixture_path, 'manifests')
end

[logicminds]

Curious if this had any performance hit? This is really helpful.

[gCardinal]

I didn't really notice any. There must be some performance lost, we're clearing the cache on each call after all, but I can't say it's slow now.

Then again I am pretty new at all this, so it's not like I have tons of hands on experience.

Right now I have 55 examples running in 11 seconds.

[shulima]

I was trying to figure out why enabling hiera lookup through shared context with metadata wasn't working as intended. Turned out that depending on test order, catalog cached from the hiera-enabled test was polluting the tests that were specifically supposed not to see hiera. Thank you for the extremely helpful workaround.

[jhoblitt]

Kudos to @ngiger !

[pall-valmundsson]

I came across this issue not because catalog reuse but because of memory consumption of the cache.
The above modifications have performance issues when running many examples against the same catalog as the catalog has to be recompiled for each example (it's noticeable for large catalogs). My version tries to re-use the cache for serially run test cases but purges the cache as soon as new arguments are sent to build_catalog.

This is not heavily tested :)

module RSpec::Puppet
  module Support
    def build_catalog(*args)
      if @@cache.has_key?(args)
        @@cache[args]
      else
        @@cache = {}
        @@cache[args] ||= self.build_catalog_without_cache(*args)
      end
    end
  end
end

[domcleal]

@pall-valmundsson I've hopefully addressed the memory issue in #329 by limiting the number of cached catalogues, while still keeping multiple in memory.

The Hiera issue here's still valid I think, the only invalidation at the moment is based on :hiera_config, not :hiera_data.