icloud and keyring not saved #871

SandraCHC · 2024-06-17T17:51:05Z

Steps to Reproduce

System
Tested on Debian Bookworm and Alpine

Solution to install
I have downloaded this two files, copied to /usr/local/bin and get correct chmod
icloudpd-1.20.0-linux-amd64
icloud-1.20.0-linux-amd64

My system:

# arch
x86_64

Reproduce

icloud --username [email protected]

In your example the system should ask for the password. But I get this message:

icloud: error: No password supplied

If I use

icloud --username [email protected] --password "mypassword"

I can enter the MFA Code. But I think Keyring don't work

icloudpd --username [email protected] --list-libraries --password-provider keyring

I get this Traceback:

2024-06-17 17:45:37 DEBUG    Authenticating...
Traceback (most recent call last):
  File "starters/icloudpd.py", line 6, in <module>
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "icloudpd/base.py", line 499, in main
  File "icloudpd/base.py", line 954, in core
  File "icloudpd/authentication.py", line 57, in authenticate_
NotImplementedError: None of providers gave password
[3633] Failed to execute script 'icloudpd' due to unhandled exception!

And if I try to delete the keyring I get also en error

# icloud --username [email protected] --delete-from-keyring
Traceback (most recent call last):
  File "keyrings/alt/file_base.py", line 177, in delete_password
  File "configparser.py", line 924, in remove_option
configparser.NoSectionError: No section: 'pyicloud_3A_2F_2Ficloud_2Dpassword'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "starters/icloud.py", line 5, in <module>
  File "pyicloud_ipd/cmdline.py", line 186, in main
  File "pyicloud_ipd/utils.py", line 66, in delete_password_in_keyring
  File "keyring/core.py", line 66, in delete_password
  File "keyrings/alt/file_base.py", line 180, in delete_password
keyring.errors.PasswordDeleteError: Password not found
[3676] Failed to execute script 'icloud' due to unhandled exception!

Other Question:

Can i save two Keyrings and use the command icloudpd twice for both user?
Can I download the Photos inside the album Names, or is the only solution to setup a own command for each album?

The text was updated successfully, but these errors were encountered:

AndreyNikiforov · 2024-06-17T18:22:55Z

my steps:

docker run --rm -it debian:bookworm

in container:

apt-get update
apt-get install -y wget
wget https://github.com/icloud-photos-downloader/icloud_photos_downloader/releases/download/v1.20.0/icloudpd-1.20.0-linux-arm64
chmod +x ./icloudpd-1.20.0-linux-arm64
./icloudpd-1.20.0-linux-arm64 -u [email protected] -d . --recent 1

I got prompt for password, then MFA code, and then one image was downloaded.

AndreyNikiforov · 2024-06-17T18:27:17Z

Can i save two Keyrings and use the command icloudpd twice for both user?

key ring saves password per user. as long as you have different user names, you can have separate passwords saved in keyring

Can I download the Photos inside the album Names, or is the only solution to setup a own command for each album?

icloudpd downloads "All Photos" album by default. You can specify any other album to download with --album parameter, but only one at a time.

SandraCHC · 2024-06-17T18:55:01Z

Thank you
I don‘t use docker. I have installed in a LXC Container direct with debian and alpine. If you wan‘t I can test with docker.

AndreyNikiforov · 2024-06-17T19:05:20Z

Thank you I don‘t use docker. I have installed in a LXC Container direct with debian and alpine. If you wan‘t I can test with docker.

Sorry for confusing you. I used docker as a way to get access to a clean debian install. Inside the container I tested as if I were using debian terminal (as root of course).

To be clear: I was not able to repro your issue on a clean debian bookworm x64 (as a container) and icloudpd v1.20.0 linux amd64 binary.

SandraCHC · 2024-06-17T19:41:42Z

Thank you
I will try tomorrow with a docker or complete netinstall iso. I thought I use an install without docker to use a shell script to download each album in separate directory

Maybe it is the lxc template with a minimal System?
Have you specific system requirements?

AndreyNikiforov · 2024-06-17T20:49:47Z

Maybe it is the lxc template with a minimal System?
Have you specific system requirements?

icloudpd is a statically compiled binary and should work on bare bones linux system. CPU consumption should be low and may spike at start up. RAM consumption may grow with the size of your library. No efforts were put in any optimization as it was never reported as a problem.

SandraCHC · 2024-06-18T04:57:37Z

I will test it now, bit I habe see you use icloudpd

I have tried with icloud for the keyring: https://github.com/icloud-photos-downloader/icloud_photos_downloader/blob/master/FAQ.md#system-keyring
(I had a tipo in my first post sorry. I have fix it to icloud)

SandraCHC · 2024-06-18T05:54:50Z

Now I have tried with a fresh Debian:bookworm Docker

docker run --rm -it debian:bookworm
apt-get update && apt-get install -y wget
wget https://github.com/icloud-photos-downloader/icloud_photos_downloader/releases/download/v1.20.0/icloudpd-1.20.0-linux-amd64
wget https://github.com/icloud-photos-downloader/icloud_photos_downloader/releases/download/v1.20.0/icloud-1.20.0-linux-amd64
chmod +x ./icloud*

Now the Commands with output:

Add data to keyring
From your FAQ I expect that I get a prompt to insert the password like icloudpd does

# ./icloud-1.20.0-linux-amd64 --username [email protected]
usage: icloud [-h] [--username USERNAME] [--password PASSWORD] [-n] [--delete-from-keyring] [--list] [--llist] [--locate] [--device DEVICE_ID] [--sound] [--message MESSAGE] [--silentmessage SILENTMESSAGE]
              [--lostmode] [--lostphone LOST_PHONE] [--lostpassword LOST_PASSWORD] [--lostmessage LOST_MESSAGE] [--outputfile] [--domain DOMAIN]
icloud: error: No password supplied

# ./icloud-1.20.0-linux-amd64 --username [email protected] --password "secret"

Two-step authentication required.
Please enter validation code
(string) --> XXXX

#

Try to list libraries with keyring

# ./icloudpd-1.20.0-linux-amd64 --username [email protected] --list-libraries --password-provider keyring
2024-06-18 05:50:51 DEBUG    Authenticating...
Traceback (most recent call last):
  File "starters/icloudpd.py", line 6, in <module>
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "icloudpd/base.py", line 499, in main
  File "icloudpd/base.py", line 954, in core
  File "icloudpd/authentication.py", line 57, in authenticate_
NotImplementedError: None of providers gave password
[2672] Failed to execute script 'icloudpd' due to unhandled exception!

Only for a test to delete keyring:

# ./icloud-1.20.0-linux-amd64 --username [email protected] --delete-from-keyring

Traceback (most recent call last):
  File "keyrings/alt/file_base.py", line 177, in delete_password
  File "configparser.py", line 924, in remove_option
configparser.NoSectionError: No section: 'pyicloud_3A_2F_2Ficloud_2Dpassword'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "starters/icloud.py", line 5, in <module>
  File "pyicloud_ipd/cmdline.py", line 186, in main
  File "pyicloud_ipd/utils.py", line 66, in delete_password_in_keyring
  File "keyring/core.py", line 66, in delete_password
  File "keyrings/alt/file_base.py", line 180, in delete_password
keyring.errors.PasswordDeleteError: Password not found
[2682] Failed to execute script 'icloud' due to unhandled exception!

Your example direct and without keyring is working.

# ./icloudpd-1.20.0-linux-amd64 --username [email protected] --list-libraries
2024-06-18 05:53:27 DEBUG    Authenticating...
iCloud Password:
2024-06-18 05:53:36 INFO     Two-factor authentication is required (2fa)
Please enter two-factor authentication code: XXXXXX
2024-06-18 05:53:45 INFO     Great, you're all set up. The script can now be run without user interaction until 2FA expires.
You can set up email notifications for when the two-factor authentication expires.
(Use --help to view information about SMTP options.)
XXXXX

AndreyNikiforov · 2024-06-18T15:34:10Z

I think I getting better idea of the problem.

icloudpd --username [email protected] --list-libraries --password-provider keyring

fails when you do not have keyring. That is expected because you specified only one password provider and it does not have password. If you specify --password-provider keyring --password-provider console then it will check keyring first and then console in case there is no password in keyring. See doc

./icloud-1.20.0-linux-amd64 --username [email protected]

exception looks like a recently introduced bug. I'll take a look at it. As a work around you can try using prior version of icloud or rely on icloudpd only (if your workflow allows that)

AndreyNikiforov · 2024-06-18T19:17:05Z

icloud behavior fixed in 1.20.1. Reopen if issue persists

petebocken · 2024-06-26T20:24:10Z

1.20.1 doesn't appear to fix this for me, see output below. Not sure what else I should provide to help troubleshoot.

Traceback (most recent call last):
  File "keyring/backends/macOS/__init__.py", line 49, in set_password
  File "keyring/backends/macOS/api.py", line 173, in set_generic_password
  File "keyring/backends/macOS/api.py", line 126, in raise_for_status
keyring.backends.macOS.api.Error: (-25308, 'Unknown Error')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "starters/icloudpd.py", line 6, in <module>
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "icloudpd/base.py", line 499, in main
  File "icloudpd/base.py", line 954, in core
  File "icloudpd/authentication.py", line 63, in authenticate_
  File "pyicloud_ipd/utils.py", line 56, in store_password_in_keyring
  File "keyring/core.py", line 61, in set_password
  File "keyring/backends/chainer.py", line 56, in set_password
  File "keyring/backends/macOS/__init__.py", line 21, in wrapper
  File "keyring/backends/macOS/__init__.py", line 53, in set_password
keyring.errors.PasswordSetError: Can't store password on keychain: (-25308, 'Unknown Error')
[82420] Failed to execute script 'icloudpd' due to unhandled exception!

petebocken · 2024-06-26T20:29:02Z

More testing...

% icloudpd --username {email-address-redacted} --password {password-redacted} --auth-only
2024-06-26 15:27:42 DEBUG    Authenticating...
Traceback (most recent call last):
  File "keyring/backends/macOS/__init__.py", line 49, in set_password
  File "keyring/backends/macOS/api.py", line 173, in set_generic_password
  File "keyring/backends/macOS/api.py", line 126, in raise_for_status
keyring.backends.macOS.api.Error: (-25308, 'Unknown Error')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "starters/icloudpd.py", line 6, in <module>
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "icloudpd/base.py", line 499, in main
  File "icloudpd/base.py", line 954, in core
  File "icloudpd/authentication.py", line 63, in authenticate_
  File "pyicloud_ipd/utils.py", line 56, in store_password_in_keyring
  File "keyring/core.py", line 61, in set_password
  File "keyring/backends/chainer.py", line 56, in set_password
  File "keyring/backends/macOS/__init__.py", line 21, in wrapper
  File "keyring/backends/macOS/__init__.py", line 53, in set_password
keyring.errors.PasswordSetError: Can't store password on keychain: (-25308, 'Unknown Error')
[84022] Failed to execute script 'icloudpd' due to unhandled exception!

petebocken · 2024-06-26T20:43:37Z

Interesting. When I try adding a new Apple account with icloudpd --username {email-address-redacted} --password {password-redacted} --auth-only It appears to work and I get the MFA notification.

So I figured maybe this other account is just messed up and needs to be resetup completely. So I deleted the cookie files in ~/.pyicloud and ran icloud --username {email-address-redacted} --delete-from-keyring it comes back with:

Traceback (most recent call last):
  File "keyring/backends/macOS/__init__.py", line 75, in delete_password
  File "keyring/backends/macOS/api.py", line 184, in delete_generic_password
  File "keyring/backends/macOS/api.py", line 117, in raise_for_status
keyring.backends.macOS.api.NotFound: (-25300, 'Item not found')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "starters/icloud.py", line 5, in <module>
  File "pyicloud_ipd/cmdline.py", line 187, in main
  File "pyicloud_ipd/utils.py", line 64, in delete_password_in_keyring
  File "keyring/core.py", line 66, in delete_password
  File "keyring/backends/chainer.py", line 63, in delete_password
  File "keyring/backends/macOS/__init__.py", line 21, in wrapper
  File "keyring/backends/macOS/__init__.py", line 77, in delete_password
keyring.errors.PasswordDeleteError: Can't delete password in keychain: (-25300, 'Item not found')
[85192] Failed to execute script 'icloud' due to unhandled exception!

AndreyNikiforov · 2024-06-27T15:15:53Z

@petebocken errors look like macOS-specific issues. There are reports about similar issue for other apps on macOS, but I did not find root cause, solution, or work around.

Old #469 in this repo seems to solve similar issue somehow.

petebocken · 2024-06-27T16:09:43Z

@AndreyNikiforov it appears it's completely hosed, I can't get anything to work. Just confirming, icloudpd should work with python 3.12, correct? I tried uninstalling all versions but that and no luck. I had 3.9, 3.10, 3.11, and 3.12 installed initially.

I'm trying to get the binary to work for now, but not sure exactly if I'm doing it right:

% ./icloudpd-ex-1.20.1-macos-amd64 auth add {email-address-redacted}
Traceback (most recent call last):
  File "starters/icloudpd_ex.py", line 110, in <module>
  File "starters/icloudpd_ex.py", line 106, in main
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
TypeError: add() got an unexpected keyword argument 'appleid'
[32070] Failed to execute script 'icloudpd_ex' due to unhandled exception!

AndreyNikiforov · 2024-06-27T16:40:24Z

@AndreyNikiforov it appears it's completely hosed, I can't get anything to work. Just confirming, icloudpd should work with python 3.12, correct? I tried uninstalling all versions but that and no luck. I had 3.9, 3.10, 3.11, and 3.12 installed initially.

I'm trying to get the binary to work for now, but not sure exactly if I'm doing it right:
% ./icloudpd-ex-1.20.1-macos-amd64 auth add {email-address-redacted}
Traceback (most recent call last):
  File "starters/icloudpd_ex.py", line 110, in <module>
  File "starters/icloudpd_ex.py", line 106, in main
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
TypeError: add() got an unexpected keyword argument 'appleid'
[32070] Failed to execute script 'icloudpd_ex' due to unhandled exception!

try regular, not experimental, version icloudpd-1.20.1-macos-amd64. You do not need python for binary packages.

petebocken · 2024-06-27T18:35:09Z

@AndreyNikiforov 1.19.1 works for me, but not 1.20.1. I was also able to pop the binary path into my script that I always use and it ran just fine and downloaded photos. This makes sense to me since everything stopped working as of 1.20/1.20.1. So doesn't seem like it's particularly a Mac issue or my Mac exactly? Something was introduced in 1.20/1.20.1 that broke this for me.

% ./icloudpd-1.19.1-macos-amd64 --auth-only
iCloud username/email: {email-redacted}
2024-06-27 13:29:05 DEBUG    Authenticating...
iCloud Password:
2024-06-27 13:29:11 ERROR    Authentication required for Account. (421)
2024-06-27 13:29:12 INFO     Two-factor authentication is required (2fa)
Please enter two-factor authentication code: 
2024-06-27 13:29:31 INFO     Great, you're all set up. The script can now be run without user interaction until 2FA expires.
You can set up email notifications for when the two-factor authentication expires.
(Use --help to view information about SMTP options.)
2024-06-27 13:29:31 INFO     Authentication completed successfully




% ./icloudpd-1.20.1-macos-amd64 --auth-only
iCloud username/email: {email-redacted}
2024-06-27 13:30:11 DEBUG    Authenticating...
iCloud Password:
Traceback (most recent call last):
  File "keyring/backends/macOS/__init__.py", line 49, in set_password
  File "keyring/backends/macOS/api.py", line 173, in set_generic_password
  File "keyring/backends/macOS/api.py", line 126, in raise_for_status
keyring.backends.macOS.api.Error: (-25308, 'Unknown Error')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "starters/icloudpd.py", line 6, in <module>
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "icloudpd/base.py", line 499, in main
  File "icloudpd/base.py", line 954, in core
  File "icloudpd/authentication.py", line 63, in authenticate_
  File "pyicloud_ipd/utils.py", line 56, in store_password_in_keyring
  File "keyring/core.py", line 61, in set_password
  File "keyring/backends/chainer.py", line 56, in set_password
  File "keyring/backends/macOS/__init__.py", line 21, in wrapper
  File "keyring/backends/macOS/__init__.py", line 53, in set_password
keyring.errors.PasswordSetError: Can't store password on keychain: (-25308, 'Unknown Error')
[40505] Failed to execute script 'icloudpd' due to unhandled exception!

petebocken · 2024-07-12T13:53:45Z

Another update so far...

I switched to using the binaries since the pip version continues to not work after 1.19.1.

Binaries have been working up until 1.21.0. I get the error below now. 1.20.4 binary works fine.

Traceback (most recent call last):
  File "keyring/backends/macOS/__init__.py", line 49, in set_password
  File "keyring/backends/macOS/api.py", line 163, in set_generic_password
  File "keyring/backends/macOS/api.py", line 184, in delete_generic_password
  File "keyring/backends/macOS/api.py", line 126, in raise_for_status
keyring.backends.macOS.api.Error: (-25244, 'Unknown Error')

#!/bin/bash
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "starters/icloudpd.py", line 6, in <module>
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "icloudpd/base.py", line 645, in main
  File "icloudpd/base.py", line 1079, in core
  File "icloudpd/authentication.py", line 74, in authenticate_
  File "pyicloud_ipd/utils.py", line 54, in store_password_in_keyring
  File "keyring/core.py", line 61, in set_password
  File "keyring/backends/chainer.py", line 56, in set_password
  File "keyring/backends/macOS/__init__.py", line 21, in wrapper
  File "keyring/backends/macOS/__init__.py", line 53, in set_password
keyring.errors.PasswordSetError: Can't store password on keychain: (-25244, 'Unknown Error')
[86825] Failed to execute script 'icloudpd' due to unhandled exception!

AndreyNikiforov · 2024-07-12T18:18:44Z

I think I found the problem.

In 1.20.0 we introduced pluggable password providers. As part of that change, keyring provider got a capability to save validated password to key chain - in 1.19 and before saving password to keychain was only possible from separate icloud utility.

There is some unresolved error in macos when saving password to keychain throws an error. Seems that it is not only related to icloudpd but other apps see that too. I spent some time, but could not get to the bottom of it. Some experiments suggest there there may be some implicit asynchrony in keychain operations that causes failures (checking and then saving). As a workaround I'll be muting errors in 1.22.0+.

SandraCHC added the bug label Jun 17, 2024

AndreyNikiforov self-assigned this Jun 17, 2024

SandraCHC mentioned this issue Jun 18, 2024

sync-icloud.sh initialize won't ask for my password - how do I input my password? boredazfcuk/docker-icloudpd#583

Closed

AndreyNikiforov closed this as completed Jun 18, 2024

SandraCHC mentioned this issue Jun 20, 2024

[BUG] Can't sync my icloud library : Password is not stored in keyring. Please save the password in keyring. mandarons/icloud-docker#216

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

icloud and keyring not saved #871

icloud and keyring not saved #871

SandraCHC commented Jun 17, 2024 •

edited

Loading

AndreyNikiforov commented Jun 17, 2024

AndreyNikiforov commented Jun 17, 2024

SandraCHC commented Jun 17, 2024

AndreyNikiforov commented Jun 17, 2024

SandraCHC commented Jun 17, 2024

AndreyNikiforov commented Jun 17, 2024

SandraCHC commented Jun 18, 2024 •

edited

Loading

SandraCHC commented Jun 18, 2024 •

edited

Loading

AndreyNikiforov commented Jun 18, 2024

AndreyNikiforov commented Jun 18, 2024

petebocken commented Jun 26, 2024

petebocken commented Jun 26, 2024

petebocken commented Jun 26, 2024 •

edited

Loading

AndreyNikiforov commented Jun 27, 2024

petebocken commented Jun 27, 2024 •

edited

Loading

AndreyNikiforov commented Jun 27, 2024

petebocken commented Jun 27, 2024

petebocken commented Jul 12, 2024

AndreyNikiforov commented Jul 12, 2024

icloud and keyring not saved #871

icloud and keyring not saved #871

Comments

SandraCHC commented Jun 17, 2024 • edited Loading

Steps to Reproduce

Reproduce

AndreyNikiforov commented Jun 17, 2024

AndreyNikiforov commented Jun 17, 2024

SandraCHC commented Jun 17, 2024

AndreyNikiforov commented Jun 17, 2024

SandraCHC commented Jun 17, 2024

AndreyNikiforov commented Jun 17, 2024

SandraCHC commented Jun 18, 2024 • edited Loading

SandraCHC commented Jun 18, 2024 • edited Loading

AndreyNikiforov commented Jun 18, 2024

AndreyNikiforov commented Jun 18, 2024

petebocken commented Jun 26, 2024

petebocken commented Jun 26, 2024

petebocken commented Jun 26, 2024 • edited Loading

AndreyNikiforov commented Jun 27, 2024

petebocken commented Jun 27, 2024 • edited Loading

AndreyNikiforov commented Jun 27, 2024

petebocken commented Jun 27, 2024

petebocken commented Jul 12, 2024

AndreyNikiforov commented Jul 12, 2024

SandraCHC commented Jun 17, 2024 •

edited

Loading

SandraCHC commented Jun 18, 2024 •

edited

Loading

SandraCHC commented Jun 18, 2024 •

edited

Loading

petebocken commented Jun 26, 2024 •

edited

Loading

petebocken commented Jun 27, 2024 •

edited

Loading