Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect customization_specs #614

Merged
merged 2 commits into from
Sep 17, 2020
Merged

Collect customization_specs #614

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
16d3f94
Collect customization_specs
agrare Aug 18, 2020
f6ecfb0
Fetch the customization spec when provisioning
agrare Aug 25, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,8 +187,12 @@ def parse_updates(vim, parser, updated_objects)
parser.parse_ext_management_system(ems, vim.serviceContent.about)

updated_objects.each do |managed_object, update_kind, cached_props|
props = cached_props

uncached_props = retrieve_uncached_props(managed_object)
props = uncached_props.present? ? cached_props.deep_merge(uncached_props) : cached_props
props = props.deep_merge(uncached_props) if uncached_props.present?

retrieve_extra_props(managed_object, props)

parser.parse(managed_object, update_kind, props)
rescue => err
Expand Down Expand Up @@ -265,6 +269,22 @@ def uncached_prop_set(obj)
@uncached_prop_set[obj.class.wsdl_name]
end

def retrieve_extra_props(obj, cached_props)
case obj.class.wsdl_name
when "CustomizationSpecManager"
retrieve_customization_spec(obj, cached_props)
end
end

def retrieve_customization_spec(spec_manager, cached_props)
cached_props[:info].to_a.each do |spec_info|
spec_info.props[:spec] = spec_manager.GetCustomizationSpec(:name => spec_info.name)&.spec
rescue RbVmomi::Fault => err
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the failure occur in the middle of the loop, does this move on or end the loop?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also ewww N+1...There's no way to bulk ask this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would move on to the next one not skip, and no unfortunately not you can't get multiple customization scripts at once.

Copy link
Member Author

@agrare agrare Aug 21, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done here previously, the other option is to get the full spec at provision time which is closer to what the vsphere client does

# Don't fail the refresh for issues with specific items
_log.warn("Failed to get customization spec for [#{spec_info.name}]: #{err}")
end
end

def parse_content_libraries(parser)
require 'vsphere-automation-content'
require 'vsphere-automation-cis'
Expand Down
19 changes: 4 additions & 15 deletions app/models/manageiq/providers/vmware/infra_manager/inventory/collector/property_collector.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,15 @@ def destroy_property_filter(property_filter)
def ems_inventory_filter_spec(vim)
RbVmomi::VIM.PropertyFilterSpec(
:objectSet => [
extension_manager_object_spec(vim.serviceContent.extensionManager),
root_folder_object_spec(vim.serviceContent.rootFolder),
license_manager_object_spec(vim.serviceContent.licenseManager),
RbVmomi::VIM.ObjectSpec(:obj => vim.serviceContent.customizationSpecManager),
RbVmomi::VIM.ObjectSpec(:obj => vim.serviceContent.extensionManager),
RbVmomi::VIM.ObjectSpec(:obj => vim.serviceContent.rootFolder, :selectSet => root_folder_select_set),
RbVmomi::VIM.ObjectSpec(:obj => vim.serviceContent.licenseManager),
],
:propSet => ems_inventory_prop_set
)
end

def extension_manager_object_spec(extension_manager)
RbVmomi::VIM.ObjectSpec(:obj => extension_manager)
end

def ems_inventory_prop_set
property_set_from_file("ems_inventory")
end
Expand All @@ -41,18 +38,10 @@ def property_set_from_file(file_name)
end
end

def root_folder_object_spec(root)
RbVmomi::VIM.ObjectSpec(:obj => root, :selectSet => root_folder_select_set)
end

def root_folder_select_set
traversal_spec_from_file("root_folder")
end

def license_manager_object_spec(license_manager)
RbVmomi::VIM.ObjectSpec(:obj => license_manager)
end

def traversal_spec_from_file(file_name)
engine_root = ManageIQ::Providers::Vmware::Engine.root
hash = YAML.load_file(engine_root.join("config", "traversal_specs", "#{file_name}.yml"))
Expand Down
25 changes: 25 additions & 0 deletions app/models/manageiq/providers/vmware/infra_manager/inventory/parser.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,20 @@ def parse_compute_resource(object, kind, props)
end
alias parse_cluster_compute_resource parse_compute_resource

def parse_customization_spec_manager(_object, kind, props)
return if kind == "leave"

props[:info].to_a.each do |spec_info|
persister.customization_specs.build(
:name => spec_info[:name],
:typ => spec_info[:type],
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be :type

Copy link
Member Author

@agrare agrare Oct 20, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:type is a reserved column name used by STI, :typ is correct here used to not conflict with STI.

Here is the database table definition: https://github.com/ManageIQ/manageiq-schema/blob/master/db/migrate/20130923182042_collapsed_initial_migration.rb#L303

:description => spec_info[:description],
:last_update_time => spec_info[:last_update_time].to_s,
:spec => rbvmomi_to_basic_types(spec_info[:spec]).deep_stringify_keys
)
end
end

def parse_datacenter(object, kind, props)
persister.ems_folders.targeted_scope << object._ref
return if kind == "leave"
Expand Down Expand Up @@ -393,4 +407,15 @@ def lazy_find_managed_object(managed_object)
parent_collection = persister.vim_class_to_collection(managed_object)
parent_collection.lazy_find(managed_object._ref)
end

def rbvmomi_to_basic_types(obj)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we could just do obj.to_h and the RbVmomi classes would support this, going to work on an rbvmomi PR to add this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My change for this actually got merged but hasn't been released yet, I'll update this when a release is cut

case obj
when RbVmomi::BasicTypes::DataObject
obj.props.transform_values { |v| rbvmomi_to_basic_types(v) }
when Array
obj.map { |v| rbvmomi_to_basic_types(v) }
else
obj
end
end
end
32 changes: 18 additions & 14 deletions app/models/manageiq/providers/vmware/infra_manager/provision/customization.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,11 @@ def build_customization_spec

# If an existing VC customization spec was selected connect to VC and get the spec
custom_spec_name = get_option(:sysprep_custom_spec).to_s.strip
custom_spec_name = nil if custom_spec_name == "__VC__NONE__"

sysprep_spec_override = get_option(:sysprep_spec_override)
spec = load_customization_spec(custom_spec_name)
spec = spec.spec unless spec.nil?
spec = load_customization_spec(custom_spec_name) if custom_spec_name

_log.info "Loaded custom spec [#{custom_spec_name}]. Override flag: [#{sysprep_spec_override}]"
if spec && sysprep_spec_override == false
adjust_nicSettingMap(spec)
Expand Down Expand Up @@ -197,18 +199,20 @@ def validate_customization_spec(custom_spec)
end

def load_customization_spec(custom_spec_name)
custom_spec_name = nil if custom_spec_name == "__VC__NONE__"
unless custom_spec_name.blank?
_log.info "Using customization spec [#{custom_spec_name}]"
cs = source.ext_management_system.customization_specs.find_by_id(custom_spec_name)
cs = source.ext_management_system.customization_specs.find_by(:name => custom_spec_name) if cs.nil?
raise MiqException::MiqProvisionError, "Customization Specification [#{custom_spec_name}] does not exist." if cs.nil?
raise MiqException::MiqProvisionError, "Customization Specification [#{custom_spec_name}] for OS type [#{cs[:typ]}] does not match the template VM OS" if cs[:typ].downcase != source.platform
_log.info "Using customization spec [#{cs.name}]"
return cs
else
_log.info "Customization spec name is empty, no spec will be loaded."
return nil
_log.info("Using customization spec [#{custom_spec_name}]")
cs = source.ext_management_system.customization_specs.find_by(:id => custom_spec_name)
cs = source.ext_management_system.customization_specs.find_by(:name => custom_spec_name) if cs.nil?

raise MiqException::MiqProvisionError, "Customization Specification [#{custom_spec_name}] does not exist." if cs.nil?
raise MiqException::MiqProvisionError, "Customization Specification [#{custom_spec_name}] for OS type [#{cs[:typ]}] does not match the template VM OS" if cs[:typ].downcase != source.platform

_log.info("Using customization spec [#{cs.name}]")

source.ext_management_system.with_provider_connection do |vim|
vim.getVimCustomizationSpecManager&.getCustomizationSpec(cs.name)&.spec
rescue Handsoap::Fault => err
_log.warn("Failed to load customization spec [#{cs.name}]: #{err}")
nil
end
end

Expand Down
2 changes: 2 additions & 0 deletions config/property_specs/ems_inventory.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,3 +162,5 @@
- licenses
:ExtensionManager:
- extensionList
:CustomizationSpecManager:
- info
6 changes: 5 additions & 1 deletion spec/models/manageiq/providers/vmware/infra_manager/provision/customization_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@

context 'build_customization_spec for windows template' do
it 'loads existing spec' do
expect(prov_vm).to receive(:load_customization_spec).and_return(spec)
expect(prov_vm.build_customization_spec).to(eq(spec))
end

Expand All @@ -108,14 +109,17 @@
spec_for_compare.identity.userData.orgName = options[:sysprep_organization]
options[:sysprep_full_name] = 'sysprep_full_name_value'
spec_for_compare.identity.userData.fullName = options[:sysprep_full_name]
spec_for_compare.identity.identification = {}
spec_for_compare.identity.identification = VimHash.new

expect(prov_vm).to receive(:load_customization_spec).and_return(spec)
expect(prov_vm.build_customization_spec).to(eq(spec_for_compare))
end

it 'creates a new spec' do
custom_spec.spec = nil
options[:sysprep_full_name] = 'sysprep_full_name_value'
options[:sysprep_organization] = 'sysprep_organization_value'
expect(prov_vm).to receive(:load_customization_spec).and_return(nil)
expect(prov_vm.build_customization_spec).to(eq(new_spec))
end
end
Expand Down
Loading