Cart Checkout Flow Architectural Update (reference pull). Includes Apple Pay Integration POC

[nerrad]

Cart Checkout Flow Architectural Update

This pull is a reference pull for some proposed changes to checkout block architecture structure and flow. I reference Cart block as well because I think much of this can apply to the cart context as well considering express payment methods might get implemented in the cart at some point. So what is being proposed here could be adapted for the cart relatively trivially. However, the perspective of this pull is orientated at least initially, towards the checkout block implementation. It is derived out of the following guiding principles:

• Having a single source of "truth" for data within the checkout flow.
• Providing a consistent interface for payment method integrations (and potentially other important extensibility pieces in the future). This interface protects the integrity of the checkout process and isolates extension processing from checkout processing. Checkout handles all processing of the order, extensions react to and communicate data for checkout to pass along to the server where server side extensibility allows extensions to continue necessary processing there.
• Checkout flow state is tracked by status.
• Extensions (beginning with payment methods) are able to interact through checkout flow via the usage of emitted events.

Ultimately, this is a proposal, however it does hold up finishing off the cart and checkout blocks. Since we've already had some sign off from #1349 with regard to payment method integration, I don't think it's a requirement to get this reviewed by payment method developers before moving ahead with this, but it'd still be worthwhile having this reviewed at some point to catch any potential red flags (I'll be posting an internal post to help gather this feedback).

However, I do want feedback from this repos maintainers and team:

• Are there concerns with any of the general architectural proposals here?
• What red flags immediately pop up that are concerns?
• When reviewing in particular the plan section (which outlines how we'd implement the work in this pull and remaining todos) are there any adjustments to the plan that you forsee? Any additions that you think should be included?

Bringing it all together

Many of the component parts of the checkout flow have already been developed and implemented over the last few weeks of cart and checkout block work. For the most part, this proposal does not replace any work that has already been done. It simply integrates these disparate parts together into a cohesive flow that provides consistent status state and extensibility points (which focus initially on payment method extensions) to support iteration one of the blocks.

As a part of working out this architecture proposal, I built an example integration of Apple Pay. I was able to get it to the point of actually triggering the payment interface and asserting it received totals as expected. Essentially, this validated that the approach was workable. However, there are still a number of pieces needing complete before I can do a complete transaction. Still, the validation was enough for me to get this reference pull up for review and comments. This should help us with the final push to finishing off the blocks.

Before getting into some of the concrete details, I've created a flow chart that gives a high level overview of the flow:

I've rebased the pull into discrete commits to aid with review of work. Keep in mind that there's still a number of @todos to accomplish (which will be summarized in the plan section). As a reference pull, this will not be merged as is but instead I expect we will open separate pulls for implementing the separate pieces.

• Add new contexts for checkout (ea99485): In this commit are all the new contexts for the checkout flow.
• Various typedef additions (2e9bf11, 5df7852). This helps communicate expected shape for various interfaces.
• Updates in existing code to use new context provider (0c229db).
• Add new usePaymentMethodInterfaceHook and update the payment method registration api (25c4631). This new hook exposes the primary interface that will be provided to registered payment methods using the payment method registration api. This hook is implemented in 0a67165 for the rendering of registered payment methods.
• A number of obsolete hooks from the original iteration of the payment method interface were removed (24f340a). This is all replaced by the new flow exposed by the contexts.
• Add new useBillingData and useStoreOrder hooks (b86ab77). These are currently placeholders.
• Improved cart data store default state (4a221a5).
• A big POC was implementing the Apple Pay integration using the new payment method interface/registration (c8dc538). This also helped with validating the new flow/logic.
• Some server side logic for the Apple Pay integration was done server side in (d00ffa2).
• Finally in 96635b4 I updated the plugin zip build script to allow for dev plugin builds (which was useful for testing in various environments). This is definitely an improvement that can be extracted to it's own pull.

General concepts

Tracking flow through status.

At any point in the checkout lifecycle, components (and extensions hooked in) should be able to accurately detect the state of the checkout flow. This includes things like:

• Is something loading? What is loading?
• Is there an error? What is the error?
• Is the checkout calculating?

Using simple booleans can be fine in some cases, but in others it can lead to complicated conditionals and bug prone code (especially as we start writing behaviour that reacts to various flow state).

I decided to extract various elements of flow to specific status types that are tracked in the various contexts. As much as possible these statues are set internally in reaction to various actions so there's no implementation needed in children components. So while payment method extensions are expected to update the active payment method state to help communicate to checkout what their flow is at, the checkout status is tightly controlled via the contexts.

An example where this is beneficial is the isCalculating boolean exposed by the checkout context. This is actually a derived boolean that reflects at any moment if there is anything occurring which will affect totals. It considers a number of different in flight requests that might affect this and (hopefully) reliably only is false when there are no in flight requests waiting to resolve. Managing something like this at the component level (or giving components the ability to set this boolean) would lead to unpredictable results (for exampleone component setting this to true and another component immediately setting it to false).

So in general, whenever this is some sort of flow status needing tracked, it should be something set and exposed at the context level.

Emitting Events

Another tricky thing for extensibility, is providing opinionated, yet flexible interfaces for extensions to act on and react to specific events in the flow. For stability, it's important that the core checkout flow controls all communication to and from the server specific to checkout processing leaving extension requirements to the extension to handle. This allows for extensions to predictably interact with the checkout data as needed without impacting other extensions hooking into it.

One of the most reliably ways to implement this type of extensibility is via the usage of an events system. So in this proposal, the contexts expose a variety of event registration apis for extensions (beginning with payment methods) to provide callbacks that will get invoked with relevant data at the time of the event firing. This is especially useful for payment methods like Apple Pay (or eventually Chrome Pay etc) that have a very self-contained interface for processing the payment, yet need to be able to communicate with checkout and react to various events from the checkout when it's done something.

An example of this is express payment methods that have their own billing data/shipping data capture that needs to update based on rates provided by checkout. You can see this implemented with the ApplePay payment method.

Details

In this section I'm going to go into a bit more detail about some of the api for the proposed architectural flow.

Contexts

In the structure, I'm proposing that we use React.Context to expose various data for tracking state and flow in the checkout. In some cases the context maintains the "tree" state within the context itself (via useReducer) and in others in a global wp.data store (for data that communicates with the server).

Notices Context

This system was implemented by Mike in #1843. It essentially does three things:

• Contains and maintains a data store for keeping track of notices (using @wordpress/notices internally).
• Optionally automatically outputs a notice container and outputs notices to that container for display when they are created.
• Exposes (via the useStoreNotices) hook an api for getting and setting notices.

This system is exposed on components by wrapping them in a <StoreNoticesProvider> component.

Shipping Method Data context

The shipping method data context exposes the api interfaces for the following things (typedef ShippingMethodDataContext) via the useShippingMethodData hook:

• shippingErrorStatus: The current error status for shipping rates if present.
• dispatchErrorStatus: A function for dispatching a shipping error status. Used in combination with...
• shippingErrorTypes: An object with the various error statuses that can be dispatched (NONE, INVALID_ADDRESS, UNKNOWN)
• shippingRates: This is retrieved using the useShippingRates hook.
• shippingRatesLoading: True when shipping rates are loading.
• selectedRates: Will expose the selected rates (@todo will implement useSelectedRates hook from Update and select shipping rates dynamically #1794)
• setSelectedRates: Function for setting new selected rates (also exposed from hook in Update and select shipping rates dynamically #1794)
• shippingAddress: The current set shipping address.
• setShippingAddress: A function for setting the shipping address. This will trigger shipping rates updates.
• onShippingRateSuccess: This is a function for registering a callback to be invoked when shipping rates are retrieved successfully. Callbacks will receive the new rates as an argument.
• onShippingRateFail: This is a function for registering a callback to be invoked when shipping rates fail to be retrieved. Callbacks will receive the error status as an argument.
• onShippingRateSelectSuccess: This is a function for registering a callback to be invoked when shipping rate selection is successful.
• onShippingRateSelectFail: This is a function for registering a callback to be invoked when shipping rates selection is unsuccessful.
• needsShipping: Boolean indicating whether the contents of the order needs shipping (true) or not (false).

Payment Method Data Context

The payment method data context exposes the api interfaces for the following things (typedef PaymentMethodDataContext) via the usePaymentMethodData hook.

• setPaymentStatus: used to set the current active payment method flow status.Calling it returns an object of dispatcher methods with the following methods: started(), processing(), completed(), error(), failed( errorMessage, CartBillingAddress, extraPaymentMethodData), success(CartBillingAddress, extraPaymentMethodData). Note extra payment data would get attached to the order processing request so payment methods could hook in server side to do any processing of the payment as a part of the checkout processing.
• currentStatus: This is an object that returns helper methods for determining the current status of the active payment method flow. Includes: isPristine(), isStarted(), isProcessing(), isFinished(), hasError(), hasFailed(), isSuccessful().
• paymentStatuses: This is an object of payment method status constants (this is likely unnecessary).
• billingData: This is the current billing data tracked in the context state.
• setBillingData: This is used to set the billing data in the state.
• paymentMethodData: This is the current extra data tracked in the context state. This is arbitrary data provided by the payment method extension after it has completed payment for checkout to include in the processing request. Typically this would contain things like payment token and payment_method name.
• errorMessage: This exposes the current set error message provided by the active payment method (if present).
• activePaymentMethod: This is the current active payment method in the checkout.
• setActivePaymentMethod: This is used to set the active payment method (typically triggered by user selecting a payment method). Any statuses in this context will be relevant to the active payment method. When a new payment method is selected the payment status is reset to PRISTINE.

Checkout Context

This context is the main one. Internally via the <CheckoutContextProvider> it handles wrapping children in <ShippingMethodDataProvider> and <PaymentMethodDataProvider>. So the checkout components just need to be wrapped by <CheckoutContextProvider>.

The provider receives three props:

• activePaymentMethod: A string, this allows for initializing with a specific payment method as the active payment method. Potentially something that could be an option for the merchant.
• redirectUrl: A string, this is used to indicate where the checkout redirects to when it is complete. There is also a setter for updating this internally if needed.
• submitLabel: This allows for customizing the checkout submit button label.

Via the useCheckoutContext, the following are exposed:

• submitLabel: This is whatever label was passed via the provider.
• onSubmit: This is a callback to be invoked either by submitting the checkout button or by express payment methods to start checkout processing after they have finished their processing.
• isComplete: True when checkout has finished processing and the subscribed checkout processing callbacks have all been invoked along with a successful processing of the checkout by the server.
• isIdle: True when the checkout state has changed and there currently is no activity.
• isProcessing: True when checkout has been submitted and is processing all the subscribed callbacks for the processing event and then the server side processing.
• isCalculating: True when something in the checkout is/could result in totals being updated.
• hasError: True when the checkout has an error (this is a separate status from the previous).
• redirectUrl: The current set url that the checkout will redirect to when it is complete.
• onCheckoutCompleteSuccess: Function used to register a callback that will fire when the checkout has processed successfully and is marked complete.
• onCheckoutCompleteError: Function used to register a callback that will fire when something in the checkout processing has caused an error.
• onCheckoutProcessing: Function used to register a callback that will be invoked when checkout has been submitted but before it sends the request to the server to process. This would be the place to register a callback that does some sort of validation. If any callback returns an error object, then the processing will bail (see flow chart for details) and set the checkout to idle status with hasError to true.
• dispatchActions: This is an object with various functions for dispatching status in the checkout. It is not exposed to extensions but is for internal use only.

Hooks

I'm not going to go into detail for all the hooks as that is fairly straightforward from existing implementations. However I do want to highlight an important extension interface hook, usePaymentMethodInterface.

usePaymentMethodInterface

This hook is used to expose all the interfaces for the registered payment method components to utilize. Essentially the result from this hook is fed in as props on the registered payment components when they are setup by checkout. You can use the typedef (RegisteredPaymentMethodProps) to see what is fed to payment methods as props from this hook.

Why don't payment methods just implement this hook?

The contract is established through props fed to the payment method components via props. This allows us to avoid having to expose the hook publicly and experiment with how the props are retrieved, exposed in the future.

Payment Method Integration changes

One major impetus for this architectural flow proposal is the work in implementing an actual payment method! I picked apple pay first because I'm a sucker for pain. Ha! No actually, I picked it because I thought it'd be a good example of something with a relatively straightforward third party api that is self-contained as way of validating the strategy of checkout handling all order processing and the two way communication with the payment method component for it's processing.

It's important to acknowledge that for the most part, the registration of payment methods and the general principles from the work in #1349 has not changed. Consider this an evolution of that work which resulted in a few changes needed that you can see implemented in this pull:

• As mentioned already the activeContent element from registered payment methods will now receive props from the object exposed by usePaymentMethodInterface.
• Payment method configuration objects for registration now are expected to have an edit property value (that is a React Element). This property will be used in the editor context. I don't think we want to have any payment libraries etc loading in the editor context. So this provides a way for payment methods to provide whatever is needed for the preview in the editor context.
• I removed stepContent from the configuration objects. I think checkout itself should be responsible for the billing form. Any extra fields the payment method requires can be a part of it's activeContent element output in the designated ui in checkout (when the payment methods tab is active).
• PaymentMethodConfig (not ExpressPaymentMethodConfig), includes a needsBilling boolean config option (optional and defaults to true. This can be used to signal to checkout whether the payment method needs the billing data or not (and possibly used by checkout to determine whether to show the billing form or not - but that may be something controlled by other means as well, so just a signal).

Note: I've moved the Plan section to the first comment after this description to make it easier for editing

[nerrad]

Plan (jolly @todos)

Once the proposal is green-lit, here's a suggested outline of tasks for implementing the work in this pull and remaining work to finish off what is derived from this pull - each of these items below would be their own pull.

This list does not already include things that already have an issue.

• Move draft order creation to the cart endpoint (cart/order) and have the cart always return an orderId (it would just be 0 if there's no order). This way all our existing logic from the cart context can be re-used in checkout effortlessly. Effectively the order data shape is the same as the cart (Add endpoint for creating and processing order. #1938)
• Add a order processing endpoint that triggers server side processing of the order cart/order/process. This would recieve in the body additional data to pass on to the order processing (essentially this would be passing along paymentMethodData from any payment methods so payment methods can hook in server side to do their processing). Ideally even though this is REST endpoint, we'd re-use what is available in Woo core for checkout processing so existing extensions hooking in would continue to work (we might have an intermediary that changes the order status to pending or whatever usually happens in the order/checkout process flow). (Add endpoint for creating and processing order. #1938)
• Extract new contexts and add in their own pull (don't implement, just the contexts) (including type-defs). (see Add contexts for checkout #1915)
• Extract payment method registration changes to it's own pull (Update payment registration config to include edit components #1928)
• Extract usePaymentMethodInterface to it's own pull and implement. (Add new hooks related to payment methods and checkout and remove obsolete. #1929)
• Implement new context and delete obsolete hooks. (Add new hooks related to payment methods and checkout and remove obsolete. #1929)
• Extract (96635b4) into it's own pull (plugin build script changes) (see Update plugin zip build script with additional options. #1893)
• Extract minor fixes (4a221a5) to their own pull. (see Add defaults for cart data totals object #1896)
• Implement apple-pay integration in it's own pull as is (including server side registration).
• Add useBillingData and useStoreOrder hooks (extract from this pull) and implement with checkout context to handle processing. useStoreOrder hook could maybe just be useStoreOrderId as all we need is the order id provided by the cart (Add new hooks related to payment methods and checkout and remove obsolete. #1929)
• Implement Apple Pay completely (including completeing actual payment). Note have registration happen entirely via separately built and enqueued script so that this could be extracted to the actual payment method extension in the future (and we're dogfooding extension points).
• Implement remaining payment methods per scope for MVP.

The following are @todos extracted from this pull that might be done in any of the above pulls or (probably better) separately:

• Add checkout processing behaviour that handles bailing when a processing subscriber returns an error object. Also includes setting a notice form the error message and showing reported validation issues in the related form fields (Checkout Context and Validation: Improve onCheckoutProcessing event emitter to allow for aborted subscribers. #1939)
• Implement billing and payment method data persistence to the cart data store so checkout can obtain it for processing the order. This can be in the useBillingData hook and then implemented in the PaymentMethodDataProvider. (Checkout: Implement billing and payment method data persistence to cart store #1940)
• Consider (not required) implementing interactions with state via the exposed context so we are consistently using the same interface (stretch goal). (Checkout/Cart: Consolidate all interactions in components with context #1942)
• Finish implementing shipping rate selection into the ShippingMethodDataProvider (along with the emitters for those events). (Checkout: Finish integrating shipping store into the ShippingMethodDataProvider #1944)
• Finish up shipping rate error handling for shipping rate event emitters. Also include wiring up the checkout context increment/decrement calculating dispatcher to keep isCalculating status accurate. (Checkout: Finish integrating shipping store into the ShippingMethodDataProvider #1944)
• Consider exposing prepareTotalItems in base/hooks/use-payment-method-interface.js for use in blocks/cart-checkout/cart/full-cart/index.js to keep the logic used for preparing these items consistent. The implementation in this location of prepareTotalItems also isn't accounting for DISPLAY_PRICES_INCLUDING_TAXES and it probably should (I'm just unsure of how this data should be passed through to payment methods?). (Extract prepareTotalItems for consistent preparation of cart total items in multiple places. #1946)
• Part of the payment method is including the default country for any shipping address displayed (mostly used for express payment methods) (see usePaymentMethodInterface). Is this something that can be reliably retrieved from shippingAddress? (Expose default country for payment methods. #1947)
• Implement needsBilling config from PaymentMethodConfig into whether billing form shows or not? I think this might be a stretch goal because this might be just a signal and other things have a factor? We need minimum info for an order right? So this definitely deserves an issue. (PaymentMethodConfig indicating what fields it needs for billing (possibly validation?) #1948)

[nerrad]

Now that all the extraction is done and the todo lists above have associated issues I'm going to close this pull. I'm not deleting the branch yet though until I've extracted the apple pay stuff into its own pull for working on.