Graph refresh: links between entities

[cben]

Followup to #30, implements most entity links in get_*_graph.

Now kubernetes graph refresh passes full refresher tests!

• Except tagging by labels which is skipped.

A few things starting with _ indicate still missing functionality that is not covered by refresher_spec, will add tests later.

Mostly using lazy_find by secodary indexes on name or (namespace, name).
(enabled by ManageIQ/manageiq#15447)

• Still using @data_index for images & registries.
• Used a local hash similar to data_index for endpoint/service matching.

[cben]

@Ladas @agrare This troubles me ^^
Even without secondary refs, the indirection when manager_ref contains links to other entities, is magic when it works but scared me a bit...

Here I'd prefer to keep :namespace in the replicator data, and explicitly use it in indexes. I need to blacklist it so ActiveRecord .create doesn't complain. When I tried this before, InventoryCollection didn't let me blacklist things that are parts of manager_ref — can we relax this (or not enforce it for secondary indexes)?

[Ladas]

Yeah we can relax the condition for the secondary indexes. It make sense only for primary index, otherwise you would not be able to identify the same record later.

Yeah having lazy_link as part of the index is funky :-)

[cben]

cool, switched to index by blacklisted :namespace & name.

[cben]

This now passes referher specs, except tag mapping :-)
But it depends on WIP ManageIQ/manageiq#15447, which is why Travis fails.
@enoodle @zeari @zakiva @moolitayer @yaacov @agrare please review.

• There are still some disabled code parts (things starting with _), which means they lack tests. I'll work on this in followup PRs.
• Endpoints parsing into container_groups<->container_services many-to-many uses a local data_index-like hash. This probably can be rewritten to directly populate an InventoryCollection for the join table; I'd prefer to tackle this in a later PR.

[agrare]

👍

[simon3z]

@cben can you squash the commits? I think we're ready for merge.

[miq-bot]

Checked commit cben@79d0212 with ruby 2.2.6, rubocop 0.47.1, and haml-lint 0.20.0
3 files checked, 3 offenses detected

app/models/manageiq/providers/kubernetes/container_manager/refresh_parser.rb

• ❗ - Line 370, Col 9 - Style/LeadingCommentSpace - Missing space after #.
• ❗ - Line 520, Col 9 - Style/AccessorMethodName - Do not prefix reader method names with get_.
• ❗ - Line 529, Col 9 - Style/AccessorMethodName - Do not prefix reader method names with get_.

[cben]

Squashed & updated PR description.
But wait, another failure on Travis. => Should be fixed by #55

[cben]

@simon3z ready for merge?

• fixed a wrong call in get_nodes_graph.
• simplified lazy_find_* methods using lazy_find_by.
• snuck in a few more :attributes_blacklist => [:namespace] so I can consistently [:namespace] without having to .delete(:namespace) in some cases.