Authorization for not 2fa account failed with error 'Invalid authentication token.'

[kkaretnyk]

Overview

I have an old account without 2fa. Running icloudpd-1.17.0-windows-amd64.exe to extract photos on a non-2fa account fails with an error 'Invalid authentication token.' For accounts with 2fa, the problem is not relevant.

Steps to Reproduce

1. .Run \icloudpd-1.17.0-windows-amd64.exe --username <not 2fa account username> --password ***** --directory *******

Expected Behavior

Logs into icloud

Actual Behavior

Fails to login

2023-12-21 15:50:43 DEBUG    Authenticating...
2023-12-21 15:50:45 ERROR    Missing apple_id field (Missing apple_id field)
Traceback (most recent call last):
  File "pyicloud_ipd\base.py", line 387, in _authenticate_with_token
    "dsWebAuthToken": self.session_data.get("session_token"),
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "requests\sessions.py", line 637, in post
  File "pyicloud_ipd\base.py", line 178, in request
    if not code and data.get("serverErrorCode"):
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pyicloud_ipd\base.py", line 207, in _raise_error
    )
^^^^^^
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: Missing apple_id field (Missing apple_id field)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "starters\icloudpd.py", line 5, in <module>
  File "click\core.py", line 1157, in __call__
  File "click\core.py", line 1078, in main
  File "click\core.py", line 1434, in invoke
  File "click\core.py", line 783, in invoke
  File "icloudpd\base.py", line 325, in main
    core(
  File "icloudpd\base.py", line 757, in core
    icloud = authenticator(logger, domain)(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "icloudpd\authentication.py", line 31, in authenticate_
    icloud = pyicloud_ipd.PyiCloudService(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pyicloud_ipd\base.py", line 305, in __init__
    'clientMasteringNumber': '17DHotfix5',
^^^^^^^^^^^^^^^^^^^
  File "pyicloud_ipd\base.py", line 367, in authenticate
    headers=headers,
^^^^^^^^^^^^^^^^^^^^^
  File "pyicloud_ipd\base.py", line 393, in _authenticate_with_token
    req = self.session.post(
    ^^^^^^^^^^^^^^^^^^^^^^^^^
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid authentication token.', PyiCloudAPIResponseException('Missing apple_id field (Missing apple_id field)'))
[15308] Failed to execute script 'icloudpd' due to unhandled exception!

Context

I researched the problem a little. The request to https://idmsa.apple.com/appleauth/auth/signin?isRememberMeEnabled=true does not contain the X-Apple-Session-Token in the response, which is then used as dsWebAuthToken. Instead, there are X-Apple-Repair-Session-Token and X-Apple-OAuth-Context.

Request:

POST https://idmsa.apple.com/appleauth/auth/signin?isRememberMeEnabled=true HTTP/1.1
Host: idmsa.apple.com
User-Agent: Opera/9.52 (X11; Linux i686; U; en)
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Origin: https://www.icloud.com
Referer: https://www.icloud.com/
Content-Type: application/json
X-Apple-OAuth-Client-Id: d39ba9916b7251055b22c7f910e2ea796ee65e98b2ddecea8f5dde8d9d1a815d
X-Apple-OAuth-Client-Type: firstPartyAuth
X-Apple-OAuth-Redirect-URI: https://www.icloud.com
X-Apple-OAuth-Require-Grant-Code: true
X-Apple-OAuth-Response-Mode: web_message
X-Apple-OAuth-Response-Type: code
X-Apple-OAuth-State: auth-a3b3ea35-9f2b-11ee-8c59-f47b097349fb
X-Apple-Widget-Key: d39ba9916b7251055b22c7f910e2ea796ee65e98b2ddecea8f5dde8d9d1a815d
scnt: AAAA-kRBOTE3NUI3Q0Y1NkU4REE5MTE5NEYxMjkxOUYzRjhBMTdBM0YzRDU4ODhDMjc2NTg5OTE3RUVEM0FCMzU1QzFEMjVBQzA0MkZENzY4Qzg4NjdCQzM4QTQ3MEI2MzBERTI0Qzg3RkM5NTZENERDMjEzMkE2N0Y1QTlEMkNFRkUwQjREQzIyMzU3Q0JCMEVEQzc2QzRENTA1M0ZBMTkzODcwMjMxN0ExOTk1MDg0N0RERDdGMUQ4NDIwM0MxMDZGNzU2MUVEMjYyOThBMTBDNkU2RTNCMjI2NTA0M0JGMjE3RjQ0NEI4REU1ODRFMUM2MHwyAAABjIiQkg5pCkzFbVkJfXj2tQ3QVFfumPmwhktdxdzwGFy-iYCqLlL1NiyYGIfFAAYotLu0etefPNibP97YpSH_D3xQvp0OJYDEgwvhHa1DNr9q-3bAsg
X-Apple-ID-Session-Id: DA9175B7CF56E8DA91194F12919F3F8A17A3F3D5888C276589917EED3AB355C1D25AC042FD768C8867BC38A470B630DE24C87FC956D4DC2132A67F5A9D2CEFE0B4DC22357CBB0EDC76C4D5053FA1938702317A19950847DDD7F1D84203C106F7561ED26298A10C6E6E3B2265043BF217F444B8DE584E1C60
Cookie: acn01=30f5WvexUcPSl/97fovtdtQ7UGX12ppO7D/p+gAGKLTJCvGt; dslang=US-EN; site=USA; aasp=DA9175B7CF56E8DA91194F12919F3F8A17A3F3D5888C276589917EED3AB355C1D25AC042FD768C8867BC38A470B630DE24C87FC956D4DC2132A67F5A9D2CEFE0B4DC22357CBB0EDC76C4D5053FA1938702317A19950847DDD7F1D84203C106F7561ED26298A10C6E6E3B2265043BF217F444B8DE584E1C60
Content-Length: 107

{"accountName": "<username>", "password": "<password>", "rememberMe": true, "trustTokens": []}

Response

HTTP/1.1 200
Server: Apple
Date: Wed, 20 Dec 2023 18:43:48 GMT
Content-Type: text/html;charset=UTF-8
Connection: keep-alive
X-Apple-I-Request-ID: b5b4d903-9f67-11ee-ab81-79b655d4c0ad
X-FRAME-OPTIONS: DENY
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Content-Security-Policy: default-src 'self' ; child-src blob: ; connect-src 'self' https://www.apple.com https://appleid.cdn-apple.com https://webcourier.sandbox.push.apple.com https://xp-qa.apple.com ; font-src 'self' https://www.apple.com https://appleid.cdn-apple.com https://idmsa.apple.com https://gsa.apple.com https://idmsa.apple.com.cn https://signin.apple.com ; frame-src 'self' https://appleid.apple.com https://gsa.apple.com ; img-src 'self' https://www.apple.com https://appleid.cdn-apple.com https://*.mzstatic.com data: https://*.apple.com ; media-src data: ; object-src 'none' ; script-src 'self' https://www.apple.com https://appleid.cdn-apple.com https://idmsa.apple.com https://gsa.apple.com https://idmsa.apple.com.cn https://signin.apple.com ; style-src 'unsafe-inline' 'self' https://www.apple.com https://appleid.cdn-apple.com https://idmsa.apple.com https://gsa.apple.com https://idmsa.apple.com.cn https://signin.apple.com ;
Referrer-Policy: origin
X-BuildVersion: R4_1
scnt: AAAA-kRBOTE3NUI3Q0Y1NkU4REE5MTE5NEYxMjkxOUYzRjhBMTdBM0YzRDU4ODhDMjc2NTg5OTE3RUVEM0FCMzU1QzFEMjVBQzA0MkZENzY4Qzg4NjdCQzM4QTQ3MEI2MzBERTI0Qzg3RkM5NTZENERDMjEzMkE2N0Y1QTlEMkNFRkUwQjREQzIyMzU3Q0JCMEVEQzc2QzRENTA1M0ZBMTkzODcwMjMxN0ExOTk1MDg0N0RERDdGMUQ4NDIwM0MxMDZGNzU2MUVEMjYyOThBMTBDNkU2RTNCMjI2NTA0M0JGMjE3RjQ0NEI4REU1ODRFMUM2MHwzAAABjIiYO_jCpRBkV4VCH-vrLzJ2uEIVz_ieFdbbs21Uw5-H_I2Q9hIBTlJwIrMpABPm7WQTumw4e8wv7yH_sFqVbogmS8119H0gEmhXrQCGCZ5SlRy8qw
Set-Cookie: dslang=US-EN; Domain=apple.com; Path=/; Secure; HttpOnly
Set-Cookie: site=USA; Domain=apple.com; Path=/; Secure; HttpOnly
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache
Cache-Control: no-store
Set-Cookie: acn01=9BB30PNGJ/kheiTr9kBWaWMwigkrfZ6mYH4B+AAT5u2D7oAo; Max-Age=31536000; Expires=Thu, 19 Dec 2024 18:43:47 GMT; Domain=apple.com; Path=/; Secure; HttpOnly
X-Apple-Repair-Session-Token: LTGu11SwrabgxgarlKkb9zZ7+r3FdligIHqwjEfy7w2wBxrIlS75d9aJZCjdamOZ86YjMM+biida6KUwyckhsXixK0lyiqLoWfrl52qFg5l0bysRvZyZTdgrdGDGhUpsuTEIFw2pkSnMjZEeoJNV6EPKXcn7kdw+fCXUB14WNJaNI8OCLbtBKEwfBzw4Xth3NYG4lgP/W4apVx0HShHy83Yc6F7HuqARYmu4lB/6NWX0WqZpcUZ5Tb/QgYZ4djAmAibSblPzkR+jmuKR4zsQpqL3sohiUIVWsCJahye15NKcfZGXmv4mrp+1kUFCdAY1tKRYZhOAJuLVoN51eeVhCmzwzpDJZamWNErJ80eLvhz32DODySZOUlRU/hEkolhhoXYm/0v9bu6q/TMfVxAK4UjNQ4xaA3sY1g8Elwh7aGrP0H2BE/+Q97s2ZF4KKRqgDXSkWaIvhaho8o/Ug1DD7cUjpBvNcAq/0GzF3j+8fPFNoy2cI3eTNBx5g1OeQHXNMnntgrfhx0SHzFmghluZl8pmPoNNwqlPCuXVHQUbl5h0SipBBY60A/g9N7c8blwJee54QzokxOE6hrnOpBaiX3JmRYlEoZncOWf9SXp5FQBgEop7fGD0kFDCC3aqMnumObWfVdJLxyNJ+3rV/wwNcSf2NebxL+5wkVeqzmnjyTRsWbVsx8HoLwegiDtUDlYteW3j43Aj3jPY5G7IVzxOVgUiaE1oxX4qzWN3PjoO1p09C9MjfTZffMIiEsx6x1itG5JkYsw+zi5a3xPZKebA0Ynv4T655eHKhO9MCCXbi5jvEEpJeQUJKuNH83kN01gwuCnjALtFCMPRE8pTCWJusVvQv2ilMXxl5rhHOZ9u0lm2vRsA7QoYNLx4R+dNFUHZrGsLz4CYL2hWdFT2L3oxsTlUz1MV5gWiax3sXOJ/KRFSZ/AD7fqjurgJ9RA7DY2GRwNCXWwAE+bthvGDiw==
X-Apple-OAuth-Context: biboRsizSaohReelTc3dRU7E2XnyOAO/KCnDCJD81mXusgxNfHL9YhTzvfKwDrKvlb09Ztqxn8+mbKRZFyuLmkGujJESmtn64DCm2JgdD/DFLpDNOh8Tj4CpcSb25iQnrGe4cHe6p5PLVaZTlkakyWnvxttSG52TPsixB7tiFROuh+hbqJC9GaF4k2OsLOqND8EsLVptyz0ysn4NWXVPSQW1TQAT5u2G8lD7
Location: https://appleid.apple.com/widget/account/repair?widgetKey=d39ba9916b7251055b22c7f910e2ea796ee65e98b2ddecea8f5dde8d9d1a815d&rv=1&language=en_US_USA#!repair
X-Apple-ID-Session-Id: DA9175B7CF56E8DA91194F12919F3F8A17A3F3D5888C276589917EED3AB355C1D25AC042FD768C8867BC38A470B630DE24C87FC956D4DC2132A67F5A9D2CEFE0B4DC22357CBB0EDC76C4D5053FA1938702317A19950847DDD7F1D84203C106F7561ED26298A10C6E6E3B2265043BF217F444B8DE584E1C60
X-Apple-Auth-Attributes: Cu1ezaI6A4r1nkSbRlajRVroq0042VClEt5VDdgHfunpSZS0YDdvbiwONqBMaUGstOyqw2Oc9fWcHml9pamK5pgS3OENgnrJuR34IHm12bF9DQa9cAnDk/5wtWH5jgBgGHekl1CtEI+/ho2vSdaZLqwyiZHJccWw68CH/ib3GknpodhHK+bm7WQCMGU41s8Ubn1s3Mm1CuAeclq98JFbdJy6Sc0flVUu9z0KV6Gjazx0RHIQHTGVr1rv6xDhGiCFuxPWfBguABPm7Yb6QA8=
X-Apple-ID-Account-Country: USA
X-Apple-I-Rscd: 412
vary: accept-encoding
Content-Language: en-US-x-lvariant-USA
Content-Length: 24161


<!DOCTYPE html>
... I cut out the part with the html code because... takes up a lot of space

[AndreyNikiforov]

Can you try version prior to 1.17.0, pls? Prior to 1.17.0 icloudpd used old Apple auth protocol that stopped working recently for 2FA accounts, but it may still work for account without 2FA. For all new accounts Apple requires 2FA, so it will be very hard to reproduce your setup.

@scaraebeus does any of the payloads above ring any bells for you since you were looking at that area recently?

[scaraebeus]

It's possible the X-Apple-Repair-Session-Token could be passed in as the dsWebAuthToken. The call to /signin appears to always return a 200 success now from what I can tell - the actual response or error code seems to be held within X-Apple-I-Rscd or X-Apple-I-Ercd, respectively. I haven't seen the 412 code yet, so likely something about the account being in a certain state (not HSA2 enabled?)

I noticed there is also a Location header pointing to repair - it looks like the 412 code would likely cause the flow to go to the account repair with the Repair-Session-Token and do some magic (i.e - likely attempt to enable HSA2 on the account).

The OAuth-Context may not really be needed or relevant in this case.

I can try catching the X-Apple-Repair-Session-Token and passing it to dsWebAuthToken and see if that helps - unfortunately, I likely won't have time until next week sometime to look at this.

EDIT:
@kkaretnyk - If you log into iCloud directly with this account, do you get any messages to upgrade, or any errors, or does it just work?

[AndreyNikiforov]

closing as there are no responses from the issue reporter