Filter out duplicates during inventory collection #212
Conversation
| @@ -15,7 +15,7 @@ def availability_zones | |||
|
|
|||
| def cloud_services | |||
| return @cloud_services if @cloud_services.any? | |||
| @cloud_services = compute_service.handled_list(:services, {}, openstack_admin?) | |||
| @cloud_services = uniques(compute_service.handled_list(:services, {}, openstack_admin?)) | |||
There was a problem hiding this comment.
we can ignore graph refresh, its indexes will not allow duplicate records to be passed to persistor
mansam
force-pushed
the
filter-out-uniques-from-fog-response
branch
from
caa95f5 to
7a752b8
Compare
|
This pull request is not mergeable. Please rebase and repush. |
|
@mansam we will need to rerecord the VCRs, conflict with the storage targeted refresh? |
|
@Ladas Probably. They're currently rerecording. |
mansam
force-pushed
the
filter-out-uniques-from-fog-response
branch
from
7a752b8 to
90584e7
Compare
|
Checked commits mansam/manageiq-providers-openstack@02631e1~...90584e7 with ruby 2.3.3, rubocop 0.52.0, haml-lint 0.20.0, and yamllint 1.10.0 **
app/models/manageiq/providers/openstack/refresh_parser_common/helper_methods.rb
|
mansam
changed the title
|
Looks good to me, merging! |
|
@Ladas Yeah, specs to reproduce this are on my log of things to do. |
|
@miq-bot add_label fine/yes |
|
@mansam For backporting to Gaprindashvili branch, all VCR files are conflicting on cherry-pick. If I should take all VCR files from this PR ignoring the conflict, I can handle that. Otherwise, please create a PR for Gaprindashvili. For backporting to Fine branch, there are only 3 files that are clean cherry-pick. Please create a PR for Fine branch. |
|
Backported to Gaprindashvili via #219 |
|
Backported to Fine via ManageIQ/manageiq#16934 |
Initial fix for https://bugzilla.redhat.com/show_bug.cgi?id=1528663
There are some disadvantages that make it difficult to fix the bug by changing pagination in openstack_handle::pagination::marker.rb, since pagination can be different for each service and collection, Fog can make it difficult to get the raw response, and some Fog collections behave differently from others. To avoid these problems, this fix implements a filter for uniqueness at the point where the inventory is collected.
Long term the correct fix is to totally reimplement the way that paging is done in OpenstackHandle/Fog.