ReceiptUtility.extractTransactionId(appReceipt:) unexpectedly returns nil

[ronaldmannak]

This is a follow up of issue #33. I filed feedback FB14087679 with examples of receipts that unexpectedly return nil.

The key issue is that ReceiptUtility.extractTransactionId(appReceipt:) doesn't throw errors, so issues are impossible to debug. The issue in #33 turned out to be a Xcode receipt, which ReceiptUtility.extractTransactionId(appReceipt:) doesn't support. It would have been convenient if the method would throw a specific error for this specific issue.
The issue I'm having now is that some (but not all) TestFlight users and the App Store Review Team seem to send receipts that either can't be parsed by ReceiptUtility.extractTransactionId(appReceipt:) or are being parsed, but don't contain an app receipt.

It's unclear to me how to debug this issue.

The client code fetched the receipts using the code provided by Apple in the documentation:

if let appStoreReceiptURL = Bundle.main.appStoreReceiptURL,
           FileManager.default.fileExists(atPath: appStoreReceiptURL.path),
           let receiptData = try? Data(contentsOf: appStoreReceiptURL, options: .alwaysMapped) {
            body = receiptData.base64EncodedString(options: [])
            let receipt = String(data: receiptData, encoding: .utf8) ?? "NO RECEIPT"
}

[alexanderjordanbaker]

@ronaldmannak I checked the receipts you provided. Neither of the receipts contain in-app transactions, so the code is appropriately returning that no in-app transaction ids are present in the receipts. If the user hadn't purchased any in-app products, there won't be any transaction ids to return

[alexanderjordanbaker]

The issue in #33 turned out to be a Xcode receipt, which ReceiptUtility.extractTransactionId(appReceipt:) doesn't support.

Slight clarification here, ReceiptUtility does support Xcode receipts to my knowledge. The appropriate transaction id was returned, which for an Xcode receipt is just a placeholder/fake transaction id generated by Xcode which doesn't really exist

[ronaldmannak]

@alexanderjordanbaker thanks for the lightning fast response, I appreciate it.

Your feedback is helpful narrowing down the issue. Both users did purchase a subscription. The TestFlight user of the second receipt got his receipt rejected right after purchase but just a few hours later he was able to use the service. I know this is outside of the scope of App Store Server Library, but could it be that there's a delay between purchase and the receipt being updated with a transaction Id?
The app does listen to transaction updates and receive a callback from the App Store immediately after a purchase. My assumption was that that StoreKit automatically updates the receipt with the transaction Id on the client, but perhaps that's not the case?

[alexanderjordanbaker]

@ronaldmannak So that falls into the realm of client behavior so on that one I'd recommend updating your feedback ticket with the additional information. However, a recommendation if this is possible for your use case. Is it possible for you to use the StoreKit 2 framework instead of the deprecated framework you are using? The pain points you are encountering have been significantly improved, and if possible, I would recommend using StoreKit 2.

[ronaldmannak]

To follow up on this issue for people having the same issue, I found a working solution but it's honestly unclear to me if I'm doing this correctly. I believe the issue was that I updated my app to use StoreKit 2's SwiftUI interface for purchasing subscriptions while the server-side verification still relied on App Store Receipts I fetched from disk within the client app as shown in the first post of this thread. It seems there might be a delay between a StoreKit 2 purchase and an updated on-disk receipt that includes the new transaction Id. This caused a hard to find bug where I plus existing TestFlight users were able to use the app because it had an old original transaction Id, but the App Store Review team wasn't.
The first hint what was wrong came when new TestFlight tester reported he initially wasn't able to use the app, but after a few hours he was. Forcing to download an updated receipt right after a purchase should in theory be possible using SKReceiptRefreshRequest but that pops up an App Store login dialog. Not a great UX.
According to Apple documentation, App Store Receipts are now deprecated, and developers should use Transaction and AppTransaction instead (which @alexanderjordanbaker referred to when he mentioned StoreKit 2).
However, while App Store Server Library has an AppTransaction model (for app purchases), it doesn't have a Transaction model (for in-app purchases) and unfortunately the documentation how use App Store Server Library to validate StoreKit 2 in-app purchases is lacking. For example, the documentation linked above states The [verifyReceipt](https://developer.apple.com/documentation/appstorereceipts/verifyreceipt) endpoint is deprecated. To validate receipts on your server, follow the steps in [Validating receipts on the device](https://developer.apple.com/documentation/appstorereceipts/validating_receipts_on_the_device) on your server. I honestly have no idea what that means, and the linked document then states The receipt isn’t necessary if you use [AppTransaction](https://developer.apple.com/documentation/storekit/apptransaction) to validate the app download, or [Transaction](https://developer.apple.com/documentation/storekit/transaction) to validate in-app purchases. Great, but how do we use Transaction on the server?
Someone please correct me if I'm wrong, but this is how I think the validation should be handled with StoreKit 2:

1. Unlike previous warnings not to validate App Store receipts on device, with StoreKit 2 we now can validate purchases on-device using Transaction.
2. The only relevant piece of information in the transaction for the server is the transactionID.
3. That means that the server now only has one tasks: check with the App Store server that the transactionID is valid and (in my case) fetch the app account token. You probably also want to use Device Check for (some) extra security.
   If I'm still not understanding the workflow correctly, please correct me.

[alexanderjordanbaker]

@ronaldmannak
On device, you can get the signed JWS Transaction via https://developer.apple.com/documentation/storekit/verificationresult/3868429-jwsrepresentation

You can then decode this with the https://apple.github.io/app-store-server-library-swift/documentation/appstoreserverlibrary/signeddataverifier/verifyanddecodetransaction(signedtransaction:)

the verifyAndDecodeTransaction function which decodes the transaction. In case this wasn't clear

https://developer.apple.com/documentation/storekit/transaction
Is basically the same thing as
https://developer.apple.com/documentation/appstoreserverapi/jwstransaction

Unlike previous warnings not to validate App Store receipts on device, with StoreKit 2 we now can validate purchases on-device using Transaction.

Yes, however you must revalidate the signed JWS transaction on your server as well.

The only relevant piece of information in the transaction for the server is the transactionID.

The relevant data is the entire signed transaction as described above, not the transactionId.

That means that the server now only has one tasks: check with the App Store server that the transactionID is valid and (in my case) fetch the app account token. You probably also want to use Device Check for (some) extra security.

Generally you should be able to verify and decode the transaction on server without ever needing to call the App Store Server API.

[ronaldmannak]

@alexanderjordanbaker Thanks Alexander, that is super helpful. And no, the new workflow wasn't clear to me. To recap, all I need to pass the jwsRepresentation from the client to the server, and have the server verify the signed jws transaction using SignedDataVerifier.
So in contrast to the old method, the App Store receipt is not used, and there is no step where the server calls the App Store Server (e.g. getAllSubscriptionStatuses)

The code could then look like this:

Client app:

public final class StoreSubscriptionController {
   var jwsTransaction: String?

  // handle purchases and updates like so
  func handle(update status: Product.SubscriptionInfo.Status) {
    guard case .verified(let transaction) = status.transaction,
      case .verified(let renewalInfo) = status.renewalInfo else {
      return
    }
    if status.state == .subscribed || status.state == .inGracePeriod {
      jwsTransaction = status.transaction.jwsRepresentation
    } else {
      jwsTransaction = nil
    }
  }

  func authenticate() async throws {
    guard let body = jwsTransaction else {
      // handle error
    }
    // send jwsTx to server
  }
}

On server (in this case using HummingBird, similar to Vapor):

 private func validateJWS(jws: String, environment: Environment) async throws -> JWSTransactionDecodedPayload? {
        
        // 1. Set up JWT verifier
        let rootCertificates = try loadAppleRootCertificates(request: request)
        let verifier = try SignedDataVerifier(rootCertificates: rootCertificates, bundleId: bundleId, appAppleId: appAppleId, environment: environment, enableOnlineChecks: true)
        
        // 2. Parse JWS transaction
        let verifyResponse = await verifier.verifyAndDecodeTransaction(signedTransaction: jws)
        
        switch verifyResponse {
        case .valid(let payload):
            
            // 3. Check expiry date
            if let date = payload.expiresDate, date < Date() {
                throw HBHTTPError(.unauthorized)
            }            
            return payload
            
        case .invalid(let error):                   
            throw HBHTTPError(.unauthorized)
        }
    }

[alexanderjordanbaker]

@ronaldmannak You also might want to check https://developer.apple.com/documentation/appstoreserverapi/revocationdate to see if it has been revoked, but in general, yep

[ronaldmannak]

I summarized the info in the blog post below. Feel free to comment if there are any errors in it.

https://link.medium.com/MR3WrCXuUKb