Adds attributes comparator and test case #1282

sle-c · 2024-02-22T06:50:10Z

Description

Attempt to use url params as required keys in api body payload.
Updates hash diff logic to submit updated values instead of diff values.

How has this been tested?

Unit tests
Manual testing with use cases from Updating assets is broken in v13.0.0 #1164 including assets, draft order line item, and order changes using this script

# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  gem "minitest"
  gem 'shopify_api', path: '../src/github.com/Shopify/shopify-api-ruby'
  gem "pry-byebug"
end

require "minitest/autorun"
require "shopify_api"
require "pry-byebug"

class PutResourceTest < Minitest::Test
  def setup
    access_token = ''
    shop = '.myshopify.com'

    @test_session = ShopifyAPI::Auth::Session.new(
      shop:,
      access_token:
    )

    scopes = [
      'read_products', 'price_rules', 'read_orders', 'write_orders',
      'read_draft_orders', 'write_draft_orders', 'read_themes', 'write_themes'
    ]

    ShopifyAPI::Context.setup(
      api_key: '',
      api_secret_key: '',
      api_version: '2024-01',
      scope: scopes.join(","),
      is_private: false,
      is_embedded: true
    )

    ShopifyAPI::Context.activate_session(@test_session)
  end

  def teardown
    ShopifyAPI::Context.deactivate_session
  end

  def test_update_assets
    all_themes = ShopifyAPI::Theme.all(
      session: @test_session,
    )
    theme = all_themes.first

    asset = ShopifyAPI::Asset.all(asset: { key: "layout/theme.liquid" }, theme_id: theme.id).first
    asset.value = asset.value + "<p>Something new</p>\n"

    assert asset.save, asset.errors
  end

  def test_update_order_attributes
    orders = ShopifyAPI::Order.all(
      session: @test_session,
      status: "any",
    )

    order = orders.first

    assert order

    order.note_attributes = order.note_attributes + [
      {
        "name" => "test #{Time.now}",
        "value" => "hello #{Time.now}"
      }
    ]

    assert order.save, order.errors
  end

  def test_update_draft_order_line_item
    draft_order = ShopifyAPI::DraftOrder.find(session: @test_session, id: 1143974756657)

    first_line_item = draft_order.line_items[0].dup
    first_line_item["quantity"] = 3

    # shopify address
    draft_order.shipping_address =  {
      address1: "151 O'Connor Street",
      city: "Ottawa",
      country: "Canada",
      country_code: "CA",
      province: "Ontario",
      province_code: "ON",
      zip: "K2P2L8"
    }
    draft_order.note = "test2"
    draft_order.line_items = [first_line_item]

    assert draft_order.save
  end
end

Checklist:

My commit message follow the pattern described in here
I have performed a self-review of my own code.
I have added tests that prove my fix is effective or that my feature works.
I have updated the project documentation.
I have added a changelog line.

sle-c · 2024-02-22T20:32:25Z

lib/shopify_api/rest/base.rb

+      def build_required_attributes(http_method:)
+        required_attributes = {}
+
+        primary_key_value = send(self.class.primary_key)
+        unless primary_key_value.nil?
+          required_attributes[self.class.primary_key] = primary_key_value
        end

-        diff
+        path_ids = deduce_path_ids(http_method)
+        path_ids&.each do |path_id|
+          path_id_value = send(path_id)
+          required_attributes[path_id.to_s] = path_id_value unless path_id_value.nil?
+        end
+
+        required_attributes


this makes the assumption that the primary key and anything required path params might also be required in the body payload

That sounds like a reasonable assumption to me.

sle-c · 2024-02-22T20:42:45Z

Checking against the 2024-01 Open API spec files all the put/patch operations list their url params as required param in the body payload

Path: /admin/api/#{api_version}/apple_pay_certificates/{apple_pay_certificate_id}.json, Required Params: ["apple_pay_certificate_id"], URL Params: ["apple_pay_certificate_id"]
Path: /admin/api/#{api_version}/marketing_events/{marketing_event_id}.json, Required Params: ["marketing_event_id"], URL Params: ["marketing_event_id"]
Path: /admin/api/#{api_version}/blogs/{blog_id}/metafields/{metafield_id}.json, Required Params: ["blog_id", "metafield_id"], URL Params: ["blog_id", "metafield_id"]
Path: /admin/api/#{api_version}/payment_gateways/{payment_gateway_id}.json, Required Params: ["payment_gateway_id"], URL Params: ["payment_gateway_id"]
Path: /admin/api/#{api_version}/reports/{report_id}.json, Required Params: ["report_id"], URL Params: ["report_id"]
Path: /admin/api/#{api_version}/recurring_application_charges/{recurring_application_charge_id}/customize.json?recurring_application_charge[capped_amount]=200, Required Params: ["recurring_application_charge_id"], URL Params: ["recurring_application_charge_id"]
Path: /admin/api/#{api_version}/customers/{customer_id}/addresses/{address_id}.json, Required Params: ["customer_id", "address_id"], URL Params: ["customer_id", "address_id"]
Path: /admin/api/#{api_version}/customers/{customer_id}/addresses/set.json?address_ids[]=1053317290&amp;operation=destroy, Required Params: ["address_ids[]", "operation", "customer_id"], URL Params: ["customer_id"]
Path: /admin/api/#{api_version}/customers/{customer_id}/addresses/{address_id}/default.json, Required Params: ["customer_id", "address_id"], URL Params: ["customer_id", "address_id"]
Path: /admin/api/#{api_version}/customers/{customer_id}.json, Required Params: ["customer_id"], URL Params: ["customer_id"]
Path: /admin/api/#{api_version}/price_rules/{price_rule_id}/discount_codes/{discount_code_id}.json, Required Params: ["price_rule_id", "discount_code_id"], URL Params: ["price_rule_id", "discount_code_id"]
Path: /admin/api/#{api_version}/price_rules/{price_rule_id}.json, Required Params: ["price_rule_id"], URL Params: ["price_rule_id"]
Path: /admin/api/#{api_version}/gift_cards/{gift_card_id}.json, Required Params: ["gift_card_id"], URL Params: ["gift_card_id"]
Path: /admin/api/#{api_version}/inventory_items/{inventory_item_id}.json, Required Params: ["inventory_item_id"], URL Params: ["inventory_item_id"]
Path: /admin/api/#{api_version}/blogs/{blog_id}/articles/{article_id}.json, Required Params: ["blog_id", "article_id"], URL Params: ["blog_id", "article_id"]
Path: /admin/api/#{api_version}/themes/{theme_id}/assets.json, Required Params: ["theme_id"], URL Params: ["theme_id"]
Path: /admin/api/#{api_version}/blogs/{blog_id}.json, Required Params: ["blog_id"], URL Params: ["blog_id"]
Path: /admin/api/#{api_version}/comments/{comment_id}.json, Required Params: ["comment_id"], URL Params: ["comment_id"]
Path: /admin/api/#{api_version}/pages/{page_id}.json, Required Params: ["page_id"], URL Params: ["page_id"]
Path: /admin/api/#{api_version}/redirects/{redirect_id}.json, Required Params: ["redirect_id"], URL Params: ["redirect_id"]
Path: /admin/api/#{api_version}/script_tags/{script_tag_id}.json, Required Params: ["script_tag_id"], URL Params: ["script_tag_id"]
Path: /admin/api/#{api_version}/themes/{theme_id}.json, Required Params: ["theme_id"], URL Params: ["theme_id"]
Path: /admin/api/#{api_version}/draft_orders/{draft_order_id}.json, Required Params: ["draft_order_id"], URL Params: ["draft_order_id"]
Path: /admin/api/#{api_version}/draft_orders/{draft_order_id}/complete.json, Required Params: ["draft_order_id"], URL Params: ["draft_order_id"]
Path: /admin/api/#{api_version}/orders/{order_id}/risks/{risk_id}.json, Required Params: ["order_id", "risk_id"], URL Params: ["order_id", "risk_id"]
Path: /admin/api/#{api_version}/orders/{order_id}.json, Required Params: ["order_id"], URL Params: ["order_id"]
Path: /admin/api/#{api_version}/custom_collections/{custom_collection_id}.json, Required Params: ["custom_collection_id"], URL Params: ["custom_collection_id"]
Path: /admin/api/#{api_version}/products/{product_id}/images/{image_id}.json, Required Params: ["product_id", "image_id"], URL Params: ["product_id", "image_id"]
Path: /admin/api/#{api_version}/variants/{variant_id}.json, Required Params: ["variant_id"], URL Params: ["variant_id"]
Path: /admin/api/#{api_version}/products/{product_id}.json, Required Params: ["product_id"], URL Params: ["product_id"]
Path: /admin/api/#{api_version}/smart_collections/{smart_collection_id}.json, Required Params: ["smart_collection_id"], URL Params: ["smart_collection_id"]
Path: /admin/api/#{api_version}/smart_collections/{smart_collection_id}/order.json?products[]=921728736&amp;products[]=632910392, Required Params: ["smart_collection_id"], URL Params: ["smart_collection_id"]
Path: /admin/api/#{api_version}/checkouts/{token}.json, Required Params: ["token"], URL Params: ["token"]
Path: /admin/api/#{api_version}/collection_listings/{collection_listing_id}.json, Required Params: ["collection_listing_id"], URL Params: ["collection_listing_id"]
Path: /admin/api/#{api_version}/mobile_platform_applications/{mobile_platform_application_id}.json, Required Params: ["mobile_platform_application_id"], URL Params: ["mobile_platform_application_id"]
Path: /admin/api/#{api_version}/product_listings/{product_listing_id}.json, Required Params: ["product_listing_id"], URL Params: ["product_listing_id"]
Path: /admin/api/#{api_version}/carrier_services/{carrier_service_id}.json, Required Params: ["carrier_service_id"], URL Params: ["carrier_service_id"]
Path: /admin/api/#{api_version}/fulfillment_services/{fulfillment_service_id}.json, Required Params: ["fulfillment_service_id"], URL Params: ["fulfillment_service_id"]
Path: /admin/api/#{api_version}/shopify_payments/disputes/{dispute_id}/dispute_evidences.json, Required Params: ["dispute_id"], URL Params: ["dispute_id"]
Path: /admin/api/#{api_version}/countries/{country_id}.json, Required Params: ["country_id"], URL Params: ["country_id"]
Path: /admin/api/#{api_version}/countries/{country_id}/provinces/{province_id}.json, Required Params: ["country_id", "province_id"], URL Params: ["country_id", "province_id"]
Path: /admin/api/#{api_version}/webhooks/{webhook_id}.json, Required Params: ["webhook_id"], URL Params: ["webhook_id"]
Total paths: 230
Total put/patch operations: 42
Total paths with required parameters: 42
Total paths with matching required parameters: 42

sle-c · 2024-02-22T21:04:30Z

lib/shopify_api/utils/attributes_comparator.rb

+              if has_numbered_key && ref_value.is_a?(Array)
+                new_hash[key] = ref_value


array diff will have the following format

original = { "a" => [] } updated = { "a" => ["foo"] } diff = { "a" => { 0 => "foo" } }

However, we don't want to submit the diff values, we want to submit the array so these 2 lines just take the updated value

has_numbered_key = true # because of `0` key ref_value = ["foo"] # which is_a Array resulting_hash = { "a" => ["foo"] }

i don't dig deeper into this because when assigning a resource's array, we either pass in a completely new array, adding to it, modify an existing item in the array or removing an existing item. Then it's better to just use the entire updated array to send to Shopify.

sle-c · 2024-02-22T21:09:33Z

lib/shopify_api/utils/attributes_comparator.rb

+              else
+                new_value = build_update_value(value, path: current_path, reference_values: reference_values)
+
+                new_hash[key] = new_value unless new_value.empty? && !ref_value.empty?


this is for cases where an object is explicitly emptied. For example

draft_order = ShopifyAPI::DraftOrder.find(session: @test_session, id: 1143974756657) draft_order.shipping_address = {} draft_order.save!

if there was a shipping address in the draft order, this should remove that shipping address from the draft order.

in an opposite case, where we only update the address partially but accidentally use the same value for the same attribute, this helps clear out the entire diff because there's nothing to update. It's an edge case. For example

original = { shipping_address: { address1: "foo", address2: "bar" } } updated = { shipping_address: { address2: "bar" } } diff = { shipping_address: { address1: HashDiff::NO_VALUE # hash_diff thinks we want to remove `address1` } # address2 doesn't show up because it's not different } update_value = {} # should be nothing to update

paulomarg · 2024-02-22T21:18:09Z

lib/shopify_api/rest/base.rb

+      def build_required_attributes(http_method:)
+        required_attributes = {}
+
+        primary_key_value = send(self.class.primary_key)
+        unless primary_key_value.nil?
+          required_attributes[self.class.primary_key] = primary_key_value
        end

-        diff
+        path_ids = deduce_path_ids(http_method)
+        path_ids&.each do |path_id|
+          path_id_value = send(path_id)
+          required_attributes[path_id.to_s] = path_id_value unless path_id_value.nil?
+        end
+
+        required_attributes


That sounds like a reasonable assumption to me.

lib/shopify_api/utils/attributes_comparator.rb

test/utils/attributes_comparator_test.rb

lizkenyon

Thank you for tackling this bug! The community will appreciate this fix! 🎉

I was thinking we could add a bit of explicit information into the docs on the usage of REST resources about this.
Explaining that only changed attributes + required URL params are sent when saving.
We could add a information about explicitly setting to null values.
This could help devs debug in the future if there are some cases this solution doesn't cover perfectly.

sle-c · 2024-02-23T19:31:15Z

Doc updated

docs/usage/rest.md

Co-authored-by: Paulo Margarido <[email protected]>

sle-c added 6 commits February 22, 2024 01:49

Adds attributes comparator and test case

a83923b

Update some test cases to cover more bases

9004239

Removes byebug

8ad3986

Moves hash_diff to comparator

0971d72

Fixes resource base tests

bfbb2d4

Adds changelog entry

910e33a

sle-c commented Feb 22, 2024

View reviewed changes

sle-c marked this pull request as ready for review February 22, 2024 20:33

sle-c requested a review from a team as a code owner February 22, 2024 20:33

sle-c commented Feb 22, 2024

View reviewed changes

paulomarg reviewed Feb 22, 2024

View reviewed changes

paulomarg approved these changes Feb 23, 2024

View reviewed changes

Changes unless condition to if and add some comments to explain

c0621c7

lizkenyon approved these changes Feb 23, 2024

View reviewed changes

Updates rest.md to explain how updating resource works

327a819

sle-c added 2 commits February 23, 2024 14:32

Minor content format

930e582

Fixes typo

a09db36

paulomarg reviewed Feb 23, 2024

View reviewed changes

docs/usage/rest.md Outdated Show resolved Hide resolved

docs/usage/rest.md Outdated Show resolved Hide resolved

sle-c and others added 2 commits February 23, 2024 14:48

Update docs/usage/rest.md

221f06e

Co-authored-by: Paulo Margarido <[email protected]>

Updates rest.md to omit implementation details

938aece

sle-c merged commit 11cbfd3 into main Feb 23, 2024
5 checks passed

sle-c deleted the sle-c/fixes-resource-diff branch February 23, 2024 21:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds attributes comparator and test case #1282

Adds attributes comparator and test case #1282

sle-c commented Feb 22, 2024 •

edited

Loading

sle-c Feb 22, 2024

paulomarg Feb 22, 2024

sle-c commented Feb 22, 2024

sle-c Feb 22, 2024

sle-c Feb 22, 2024 •

edited

Loading

paulomarg Feb 22, 2024

sle-c Feb 22, 2024 •

edited

Loading

sle-c Feb 22, 2024

paulomarg Feb 22, 2024

lizkenyon left a comment

sle-c commented Feb 23, 2024

		if has_numbered_key && ref_value.is_a?(Array)
		new_hash[key] = ref_value

Adds attributes comparator and test case #1282

Adds attributes comparator and test case #1282

Conversation

sle-c commented Feb 22, 2024 • edited Loading

Description

How has this been tested?

Checklist:

sle-c Feb 22, 2024

Choose a reason for hiding this comment

paulomarg Feb 22, 2024

Choose a reason for hiding this comment

sle-c commented Feb 22, 2024

sle-c Feb 22, 2024

Choose a reason for hiding this comment

sle-c Feb 22, 2024 • edited Loading

Choose a reason for hiding this comment

paulomarg Feb 22, 2024

Choose a reason for hiding this comment

sle-c Feb 22, 2024 • edited Loading

Choose a reason for hiding this comment

sle-c Feb 22, 2024

Choose a reason for hiding this comment

paulomarg Feb 22, 2024

Choose a reason for hiding this comment

lizkenyon left a comment

Choose a reason for hiding this comment

sle-c commented Feb 23, 2024

sle-c commented Feb 22, 2024 •

edited

Loading

sle-c Feb 22, 2024 •

edited

Loading

sle-c Feb 22, 2024 •

edited

Loading