General Performance Issues

[kb6673]

All,

We have a customer store that is struggling to sync WooCommerce Orders -> MailChimp.

It's been stuck like this for weeks.

This part of the page never updates.

We've disabled the plugin and re-enabled.
We've upgraded to PHP 7.2 (We have PHP 7.1)
I've checked the failed and queued jobs tables - not really sure what I am looking for, but they look OK. failed is empty, queued jobs seem to change during the day.

We tried the manual resync. Nothing changed.

So I followed the info #158. In wp-config.php we have

// for mailchimp
define('DISABLE_WP_HTTP_WORKER', true);
define('MAILCHIMP_DISABLE_QUEUE', true);

I am then running the command wp queue listen. It takes about 30 - 40 minutes to complete

Tailing the log suggests good things are happening

Once it completes, it has managed to sync about 5 orders. Sometimes 1, sometimes 10.

The number of NOTICE order_submit.success :: updateStoreOrder :: entries in the log no where near match the number of orders that have been sycned from the MC status page.

We have over 10,000 orders left sync. At 30 minutes for 5 orders, it's going to take, err... super ages.

I spoke to MailChimp support, however they weren't too helpful apart from the usual canned responses. I suppose this is an extreme example.

One suggestion was to disable plugins and see what happens, however this store does hundreds of orders a day, so disabling plugins, leaving the site in a broken state whilst wp queue listen runs for 30 minutes isn't sustainable.

We've copied the data over to the dev site and tried syncing with minimal plugins - we get the same results.

We've read and read most of the information - people are having the same issue, however I am not sure what the solution is for most people.

Is there a solution to this? Any help would be appreciated.

Update 1

Occasionally we see what appears to be broken serialised data in the job table.

Update 2

Deleting the jobs in `wp_queue" results in the job being created again that shows the broken serialised data

Update 3

I see the following errors popup in the log quite often

2019-09-27T01:47:35+00:00 ERROR api :: processCurlResponse :: Array
(
    [info] => Array
        (
            [url] => https://us18.api.mailchimp.com/3.0/ecommerce/stores/$
            [content_type] => application/problem+json; charset=utf-8
            [http_code] => 404
            [header_size] => 631
            [request_size] => 370
            [filetime] => -1
            [ssl_verify_result] => 0
            [redirect_count] => 0
            [total_time] => 0.479852
            [namelookup_time] => 0.000203
            [connect_time] => 0.002031
            [pretransfer_time] => 0.023823
            [size_upload] => 0
            [size_download] => 200
            [speed_download] => 417
            [speed_upload] => 0
            [download_content_length] => 200
            [upload_content_length] => -1
            [starttransfer_time] => 0.479806
            [redirect_time] => 0
            [redirect_url] =>
            [primary_ip] => 104.123.89.75
            [certinfo] => Array
                (
                )

            [primary_port] => 443
            [local_ip] => 10.251.140.11
            [local_port] => 43594
            [request_header] => DELETE /3.0/ecommerce/stores/-----/carts/$
Host: us18.api.mailchimp.com
Authorization: Basic ------
Accept: */*
Accept-Encoding: deflate, gzip, br
content-type: application/json
user-agent: MailChimp for WooCommerce/2.2.4; PHP/7.2.22; WordPress/4.9.8; Woo/3.5.7;


        )

    [data] => Array
        (
            [type] => http://developer.mailchimp.com/documentation/mailchimp/guides/error-gl$
            [title] => Resource Not Found
            [status] => 404
            [detail] => The requested resource could not be found.
            [instance] => 9d239d7c-804f-4dee-b713-05c2644a289b
        )

)

Update 4

I've run the wp_queue_listen 7 times so far today and we've managed to sync 9 orders. Only 11,059 orders to go!

[ryanhungate]

@kb6673 this is interesting to hear that you're not seeing orders in Mailchimp - although the plugin logs ( and CLI logs ) say otherwise. Do you have multiple lists that you've tried to connect to? The audience that you've connected to your store should be associating all of this e-comm data per member, even if it's only transactional. Something does seem off here for sure.

Since you seem to be pretty technical - can you to execute an API call to Mailchimp to see how many orders you have in your store? I'm curious to see if the API returns a different result than what you're seeing in the UI of Mailchimp itself.

The endpoint you'll want to call is /ecommerce/stores/{store_id}/orders

If you would rather have me to this for you personally - please send a temporary API key to my email address ryan at vextras.com and i'll be more than happy to do some debugging for you myself.

[kb6673]

Hi @ryanhungate I went down a different route just before your reply. I disconnected the store in Woo. Then I disconnected the store in MC. Reconnected the store, that gave me zero values.

Coupons synced OK. Products synced OK. So far it has synced 933 orders in the last hour. That's 100x times more than it's done in two weeks.

I am getting a fairly high rate of 404 errors as shown in Update 3 above. I am not sure if we are hitting a MC API rate limit? I guess we've been bashing the API all day.

I'm going to leave this running over the weekend and see how far it gets.

---

If there is no success, I'll get you the data you've asked for.

Thanks

[ryanhungate]

@kb6673 ok sounds good. Glad you got that part done - I think disconnecting and going through the whole process deleting on the connected sites area of MC is all you needed to do. Obviously syncing a large store will take time, and yes, you will have some rate limit issues as everyone else would too - but it should just keep trucking along and get things up to speed over time.

Once the sync is complete, the daily activity is really not an issue, even if you're getting many hundreds of orders a day 👍

Let us know if anything else comes up - but for now, since we're this far with your sync, I feel safe to close this out.