Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial row updates in parallel #17861

Merged
merged 28 commits into from
Aug 27, 2018
Merged
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
4b4d106
Allow skeletal_index to be serializable
Ladas Aug 7, 2018
40d1083
Use :timestamp for the full row parallel update check
Ladas Aug 7, 2018
1fb4b97
Allow skeletal_index objects to be updated
Ladas Aug 7, 2018
1897df2
Expose data iterator into skelatal index
Ladas Aug 8, 2018
1a2415e
A basic version of partial updates
Ladas Aug 8, 2018
5afa339
Saving partial after full plus specs passing
Ladas Aug 9, 2018
d4754fe
Timestamps are set correctly when combining different partials
Ladas Aug 9, 2018
3ad9192
Correct update for full after partial
Ladas Aug 9, 2018
6792e79
Use supports_timestamps_max for conditionalization of the entire path
Ladas Aug 9, 2018
2f40ea3
Skeletonize full row if it fails timestamp check
Ladas Aug 9, 2018
f371709
Make full to partial transformation also for the upsert path
Ladas Aug 10, 2018
c57ba04
Building 2 partial InventoryObjects merges them according to timestamps
Ladas Aug 10, 2018
bf74c7d
Fix assign attributes being too strict for the full rows
Ladas Aug 10, 2018
dfdd349
Don't do partial updates if timestamp is missing
Ladas Aug 10, 2018
810adeb
Check timestamp before updating
Ladas Aug 10, 2018
a7111b9
Update TODO about timestamp
Ladas Aug 10, 2018
4add89b
Proper parallel_safe? conditions, so we don't break existing code
Ladas Aug 10, 2018
01e2a34
Rename timestamp to resource_timestamp
Ladas Aug 10, 2018
08534ab
Allow to update STI :type
Ladas Jan 31, 2018
8c3d957
resource_version should work the same as resource_timestamp
Ladas Aug 14, 2018
853ce8b
Fill correct created and updated result of IC
Ladas Aug 15, 2018
fe6c581
Cleanup TODos ans YARD
Ladas Aug 15, 2018
512a7e7
Cleanup sql_helper and separate to upsert and update mixins
Ladas Aug 15, 2018
19eccb7
Fix rubocop issues
Ladas Aug 15, 2018
2ad8435
Inventory collection logging is bad
Ladas Aug 16, 2018
67b03ee
Quote every column name plus cleanup of update query
Ladas Aug 16, 2018
c36d4bd
Store integer in resource_versions jsonb
Ladas Aug 16, 2018
4f3c6dc
Add timestamp conditions only for parallel safe strategy
Ladas Aug 22, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix assign attributes being too strict for the full rows
Fix assign attributes being too strict for the full rows
  • Loading branch information
Ladas committed Aug 27, 2018
commit bf74c7d3035e2cb2d92f83243a34a69613d81137
23 changes: 16 additions & 7 deletions app/models/manager_refresh/inventory_object.rb
Original file line number Diff line number Diff line change
@@ -169,21 +169,30 @@ def assign_attributes(attributes)

if assign
public_send("#{k}=", v) # Attribute is newer than current one, lets use it
data[:timestamps][k] = specific_attr_timestamp if specific_attr_timestamp # and set the latest timestamp
(data[:timestamps] ||= {})[k] = specific_attr_timestamp if specific_attr_timestamp # and set the latest timestamp
end
else
public_send("#{k}=", v)
end
end

if inventory_collection.supports_timestamps_max? && attributes[:timestamp] && data[:timestamp]
# If timestamps are present, store the bigger one
data[:timestamp] = attributes[:timestamp] if attributes[:timestamp] > data[:timestamp]
if inventory_collection.supports_timestamps_max?
if attributes[:timestamp] && data[:timestamp]
# If both timestamps are present, store the bigger one
data[:timestamp] = attributes[:timestamp] if attributes[:timestamp] > data[:timestamp]
elsif attributes[:timestamp] && !data[:timestamp]
# We are assigning timestamp that was missing
data[:timestamp] = attributes[:timestamp]
end
end

if inventory_collection.supports_resource_versions_max? && attributes[:timestamp] && data[:timestamp]
# If timestamps are present, store the bigger one
data[:timestamp] = attributes[:timestamp] if attributes[:timestamp] > data[:timestamp]
if inventory_collection.supports_resource_versions_max?
if attributes[:resource_version] && data[:resource_version]
# If timestamps are present, store the bigger one
data[:resource_version] = attributes[:resource_version] if attributes[:resource_version] > data[:resource_version]
elsif attributes[:resource_version] && !data[:resource_version]
data[:resource_version] = attributes[:resource_version]
end
end

self