Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot Initialize icloudpd #400

Closed
s-leeq opened this issue Oct 2, 2023 · 9 comments
Closed

Cannot Initialize icloudpd #400

s-leeq opened this issue Oct 2, 2023 · 9 comments

Comments

@s-leeq
Copy link

s-leeq commented Oct 2, 2023

Running Environment: Docker with OrbStack(1.0.0) in macOS 14.0 (23A344)
Docker Version:

Client:
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:28:49 2023
 OS/Arch:           darwin/amd64
 Context:           orbstack

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:32:17 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.7.5
  GitCommit:        fe457eb99ac0e27b3ce638175ef8e68a7d2bc373
 runc:
  Version:          1.1.9
  GitCommit:        82f18fe0e44a59034f3e1f45e475fa5636e539aa
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Error Log:

/ sync-icloud.sh --Initialise --Enable-Debugging

2023-10-02 11:13:52 INFO     ***** boredazfcuk/icloudpd container for icloud_photo_downloader v1.0.651 started *****
2023-10-02 11:13:52 INFO     ***** For support, please go here: https://github.com/boredazfcuk/docker-icloudpd *****
2023-10-02 11:13:52 INFO     Alpine Linux 3.18.3
2023-10-02 11:13:52 INFO     Python version: 3.11.5
2023-10-02 11:13:52 INFO     Loading configuration from: /config/icloudpd.conf
2023-10-02 11:14:20 INFO     Apple ID: xxx
2023-10-02 11:14:20 INFO     Authentication Type: MFA
2023-10-02 11:14:20 INFO     Cookie path: /config/xxx
2023-10-02 11:14:20 INFO     Cookie expiry notification period: 7
2023-10-02 11:14:20 INFO     Download destination directory: /home/xxx/iCloud
2023-10-02 11:14:20 INFO     Folder structure: {:%Y/%m/%d}
2023-10-02 11:14:20 INFO     Synchronisation interval: 86400
2023-10-02 11:14:20 INFO     Synchronisation delay (minutes): 0
2023-10-02 11:14:20 INFO     Set EXIF date/time: false
2023-10-02 11:14:20 INFO     Auto delete: false
2023-10-02 11:14:20 INFO     Delete after download: false
2023-10-02 11:14:20 INFO     Photo size: original
2023-10-02 11:14:20 INFO     Single pass mode: false
2023-10-02 11:14:20 INFO     Skip download check: false
2023-10-02 11:14:20 INFO     Skip live photos: false
2023-10-02 11:14:20 INFO     Number of most recently added photos to download: Download All Photos
2023-10-02 11:14:20 INFO     Downloading photos from: Download All Photos
2023-10-02 11:14:20 INFO     Stop downloading when prexisiting files count is: Download All Photos
2023-10-02 11:14:20 INFO     Live photo size: original
2023-10-02 11:14:20 INFO     Skip videos: false
2023-10-02 11:14:20 INFO     Convert HEIC to JPEG: false
2023-10-02 11:14:20 INFO     Downloading from: icloud.com.cn
2023-10-02 11:14:20 INFO     Authentication domain: cn
2023-10-02 11:14:20 INFO     Ignore Synology extended attribute directories: Disabled
2023-10-02 11:14:20 INFO     Script launch parameters: --Initialise
2023-10-02 11:14:20 WARNING  Group id, 20, already in use by the group: dialout - continuing as force_gid variable has been set. Group name to use: dialout
Traceback (most recent call last):
  File "/opt/icloudpd_v1.7.2_china/bin/icloud", line 33, in <module>
    sys.exit(load_entry_point('pyicloud==1.0.0', 'console_scripts', 'icloud')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/pyicloud/cmdline.py", line 183, in main
    password = utils.get_password(
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/pyicloud/utils.py", line 15, in get_password
    return get_password_from_keyring(username)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/pyicloud/utils.py", line 39, in get_password_from_keyring
    result = keyring.get_password(KEYRING_SYSTEM, username)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 56, in get_password
    return get_keyring().get_password(service_name, username)
           ^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 33, in get_keyring
    init_backend()
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 84, in init_backend
    set_keyring(_detect_backend(limit))
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 99, in _detect_backend
    or load_config()
       ^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 164, in load_config
    config.read(_ensure_path(_config_path()), encoding='utf-8')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 154, in _ensure_path
    if not path.exists():
           ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/pathlib.py", line 1235, in exists
    self.stat()
  File "/usr/lib/python3.11/pathlib.py", line 1013, in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/root/.config/python_keyring/keyringrc.cfg'
2023-10-02 11:14:21 ERROR    Keyring file does not exist. Please try again.

It's confusing me that while other issues reported FileExistsError: [Errno 17] File exists: '/home/user/.local/share/python_keyring', I always received PermissionError: [Errno 13] Permission denied: '/root/.config/python_keyring/keyringrc.cfg' even if I set the user_id, group_id and force_group environment variables.
@boredazfcuk
Copy link
Owner 

 sync-icloud.sh --Initialise --Enable-Debugging

Only a single command line parameter is processed by the script so --Enable-Debugging is being ignored. Can you run that separately and then post another log file?

Your script attempting to save files in /root/ suggests that your user_id variable is set to 0. I've no idea how this will work as I'm using su to switch user when saving the files. This location is also not on the persistent storage locations that are defined in the container, so will be removed each time the container is rebuilt.

@s-leeq
Copy link
Author

s-leeq commented Oct 3, 2023

It seems like sync-icloud.sh refused the --Enable-Debugging parameter, when I ran it separately, it returned:

/ # sync-icloud.sh --Enable-Debugging

2023-10-03 08:45:34 INFO     ***** boredazfcuk/icloudpd container for icloud_photo_downloader v1.0.651 started *****
2023-10-03 08:45:34 INFO     ***** For support, please go here: https://github.com/boredazfcuk/docker-icloudpd *****
2023-10-03 08:45:34 INFO     Alpine Linux 3.18.3
2023-10-03 08:45:34 INFO     Python version: 3.11.5
2023-10-03 08:45:34 INFO     Loading configuration from: /config/icloudpd.conf
2023-10-03 08:46:02 INFO     Apple ID: xxx
2023-10-03 08:46:02 INFO     Authentication Type: MFA
2023-10-03 08:46:02 INFO     Cookie path: /config/xxx
2023-10-03 08:46:02 INFO     Cookie expiry notification period: 7
2023-10-03 08:46:02 INFO     Download destination directory: /home/stan/iCloud
2023-10-03 08:46:02 INFO     Folder structure: {:%Y/%m/%d}
2023-10-03 08:46:02 INFO     Synchronisation interval: 86400
2023-10-03 08:46:02 INFO     Synchronisation delay (minutes): 0
2023-10-03 08:46:02 INFO     Set EXIF date/time: false
2023-10-03 08:46:02 INFO     Auto delete: false
2023-10-03 08:46:02 INFO     Delete after download: false
2023-10-03 08:46:02 INFO     Photo size: original
2023-10-03 08:46:02 INFO     Single pass mode: false
2023-10-03 08:46:02 INFO     Skip download check: false
2023-10-03 08:46:02 INFO     Skip live photos: false
2023-10-03 08:46:02 INFO     Number of most recently added photos to download: Download All Photos
2023-10-03 08:46:02 INFO     Downloading photos from: Download All Photos
2023-10-03 08:46:02 INFO     Stop downloading when prexisiting files count is: Download All Photos
2023-10-03 08:46:02 INFO     Live photo size: original
2023-10-03 08:46:02 INFO     Skip videos: false
2023-10-03 08:46:02 INFO     Convert HEIC to JPEG: false
2023-10-03 08:46:02 INFO     Downloading from: icloud.com.cn
2023-10-03 08:46:02 INFO     Authentication domain: cn
2023-10-03 08:46:02 INFO     Ignore Synology extended attribute directories: Disabled
2023-10-03 08:46:02 WARNING  Ignoring invalid launch parameter specified: --Enable-Debugging
2023-10-03 08:46:02 WARNING  Please do not specify the above parameter when launching the container. Continuing in 2 minutes

On the other hand, the same log shown as before when I ran with --Initialise.

@boredazfcuk
Copy link
Owner

Looks like I missed it out on the part that validates the launch parameters. Fixed in latest version.

@s-leeq
Copy link
Author

s-leeq commented Oct 4, 2023

Got the same issue again in v1.0.661 :(, and I also tested in a Linux machine, still the same issue.

@boredazfcuk
Copy link
Owner

boredazfcuk commented Oct 4, 2023 •
edited
Loading

Odd, When I tested it, it worked.

All it does is edit /config/icloudpd.conf and sets debug_logging=true, can you do that manually?

@s-leeq
Copy link
Author

s-leeq commented Oct 5, 2023

I set debug_logging=true and it still didn't work:(

It seems like the variable config_dir wasn't declared in the script. It was /config when declaring config_file, then during run_as "${icloudpd_path}/icloud --username ${apple_id}" in ConfigurePassword(), it became /root/.configwhen I use a non root user and group.

The complete log:

❯ docker create \
   --name icloudpd \
   --hostname icloudpd \
   --network icloudpd_bridge \
   --restart=always \
   --env TZ=Asia/Shanghai \
   --env icloud_china=true \
   --env auth_china=true \
   --env apple_id=xxx \
   --env user=xxx \
   --env user_id=501 \
   --env group_id=20 \
   --env force_gid=true \
   --volume icloudpd_config:/config \
   --volume xxx:/home/xxx/iCloud \
   --env user=stan boredazfcuk/icloudpd
xxx
❯ docker start icloudpd
icloudpd
❯ docker exec -it icloudpd sync-icloud.sh --Initialise
2023-10-05 09:21:26 INFO     ***** boredazfcuk/icloudpd container for icloud_photo_downloader v1.0.662 started *****
2023-10-05 09:21:26 INFO     ***** For support, please go here: https://github.com/boredazfcuk/docker-icloudpd *****
2023-10-05 09:21:26 INFO     Alpine Linux 3.18.3
2023-10-05 09:21:26 INFO     Python version: 3.11.5
2023-10-05 09:21:26 INFO     Loading configuration from: /config/icloudpd.conf
2023-10-05 09:21:26 DEBUG    Running user id: 0
2023-10-05 09:21:26 DEBUG    Running group id: 0
2023-10-05 09:21:26 DEBUG    Local user: xxx:501
2023-10-05 09:21:26 DEBUG    Local group: group:20
2023-10-05 09:21:26 DEBUG    Force GID: false
2023-10-05 09:21:26 DEBUG    LAN IP Address: xxx
2023-10-05 09:21:26 DEBUG    Default gateway: xxx
2023-10-05 09:21:26 DEBUG    DNS server: 127.0.0.11
2023-10-05 09:21:26 DEBUG    IP address for icloud.com.cn: 103.115.248.1
2023-10-05 09:21:54 DEBUG    Route check to icloud.com.cn successful
2023-10-05 09:21:54 DEBUG    Apple ID: (hidden)
2023-10-05 09:21:54 INFO     Authentication Type: MFA
2023-10-05 09:21:54 DEBUG    Cookie path: /config/(hidden)
2023-10-05 09:21:54 INFO     Cookie expiry notification period: 7
2023-10-05 09:21:54 INFO     Download destination directory: /home/xxx/iCloud
2023-10-05 09:21:54 INFO     Folder structure: {:%Y/%m/%d}
2023-10-05 09:21:54 DEBUG    Directory permissions: 750
2023-10-05 09:21:54 DEBUG    File permissions: 640
2023-10-05 09:21:54 INFO     Synchronisation interval: 86400
2023-10-05 09:21:54 INFO     Synchronisation delay (minutes): 0
2023-10-05 09:21:54 INFO     Set EXIF date/time: false
2023-10-05 09:21:54 INFO     Auto delete: false
2023-10-05 09:21:54 INFO     Delete after download: false
2023-10-05 09:21:54 INFO     Photo size: original
2023-10-05 09:21:54 INFO     Single pass mode: false
2023-10-05 09:21:54 INFO     Skip download check: false
2023-10-05 09:21:54 INFO     Skip live photos: false
2023-10-05 09:21:54 INFO     Number of most recently added photos to download: Download All Photos
2023-10-05 09:21:54 INFO     Downloading photos from: Download All Photos
2023-10-05 09:21:54 INFO     Stop downloading when prexisiting files count is: Download All Photos
2023-10-05 09:21:54 INFO     Live photo size: original
2023-10-05 09:21:54 INFO     Skip videos: false
2023-10-05 09:21:54 INFO     Convert HEIC to JPEG: false
2023-10-05 09:21:54 INFO     Downloading from: icloud.com.cn
2023-10-05 09:21:54 INFO     Authentication domain: cn
2023-10-05 09:21:54 DEBUG    Nextcloud synchronisation trigger: Disabled
2023-10-05 09:21:54 DEBUG    Nextcloud upload: Disabled
2023-10-05 09:21:54 INFO     Ignore Synology extended attribute directories: Disabled
2023-10-05 09:21:54 INFO     Script launch parameters: --Initialise
2023-10-05 09:21:54 WARNING  Group id, 20, already in use by the group: dialout - continuing as force_gid variable has been set. Group name to use: dialout
2023-10-05 09:21:54 DEBUG    User, xxx:501, already created
2023-10-05 09:21:54 DEBUG    Set owner and group on icloudpd temp directory
2023-10-05 09:21:54 DEBUG    Set owner and group on config directory
2023-10-05 09:21:54 DEBUG    Configure password
2023-10-05 09:21:54 DEBUG    Adding password to keyring file: /config/python_keyring/keyring_pass.cfg
2023-10-05 09:21:54 DEBUG    Switched to icloudpd: 1.7.2-china-auth-fix
Traceback (most recent call last):
  File "/opt/icloudpd_v1.7.2_china/bin/icloud", line 33, in <module>
    sys.exit(load_entry_point('pyicloud==1.0.0', 'console_scripts', 'icloud')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/pyicloud/cmdline.py", line 183, in main
    password = utils.get_password(
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/pyicloud/utils.py", line 15, in get_password
    return get_password_from_keyring(username)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/pyicloud/utils.py", line 39, in get_password_from_keyring
    result = keyring.get_password(KEYRING_SYSTEM, username)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 56, in get_password
    return get_keyring().get_password(service_name, username)
           ^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 33, in get_keyring
    init_backend()
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 84, in init_backend
    set_keyring(_detect_backend(limit))
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 99, in _detect_backend
    or load_config()
       ^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 164, in load_config
    config.read(_ensure_path(_config_path()), encoding='utf-8')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/icloudpd_v1.7.2_china/lib/python3.11/site-packages/keyring/core.py", line 154, in _ensure_path
    if not path.exists():
           ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/pathlib.py", line 1235, in exists
    self.stat()
  File "/usr/lib/python3.11/pathlib.py", line 1013, in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/root/.config/python_keyring/keyringrc.cfg'
2023-10-05 09:21:54 ERROR    Keyring file does not exist. Please try again.

However, it kept /config if I set the user_id and group_id to 0. Then instead of

2023-10-05 09:21:54 WARNING  Group id, 20, already in use by the group: dialout - continuing as force_gid variable has been set. Group name to use: dialout

and below, new error occurred:

❯ docker create \
   --name icloudpd \
   --hostname icloudpd \
   --network icloudpd_bridge \
   --restart=always \
   --env TZ=Asia/Shanghai \
   --env icloud_china=true \
   --env auth_china=true \
   --env apple_id=xxx \
   --volume icloudpd_config:/config \
   --volume xxx:/root/iCloud \
   --env user_id=0 \
   --env group_id=0 \
   --env force_gid=true \
   boredazfcuk/icloudpd
xxx
...
2023-10-05 09:27:13 WARNING  Group id, 0, already in use by the group: root - continuing as force_gid variable has been set. Group name to use: root
2023-10-05 09:27:13 DEBUG    Creating user user:0
useradd: UID 0 is not unique
2023-10-05 09:27:13 DEBUG    Set owner and group on icloudpd temp directory
2023-10-05 09:27:13 DEBUG    Set owner and group on config directory
2023-10-05 09:27:13 DEBUG    Configure password
2023-10-05 09:27:13 DEBUG    Adding password to keyring file: /config/python_keyring/keyring_pass.cfg
2023-10-05 09:27:13 DEBUG    Switched to icloudpd: 1.7.2-china-auth-fix
su: unknown user user
2023-10-05 09:27:13 ERROR    Keyring file does not exist. Please try again.

Seems like something wrong with run_as()?

@boredazfcuk
Copy link
Owner 

   --env user_id=0 \
   --env group_id=0 \

This container is designed to be run as a non-root user. Setting these values will break things. Latest container now prevents this.

@boredazfcuk boredazfcuk reopened this Oct 5, 2023
@boredazfcuk
Copy link
Owner

boredazfcuk commented Oct 5, 2023 •
edited
Loading

It seems like the variable config_dir wasn't declared in the script. It was /config when declaring config_file, then during run_as "${icloudpd_path}/icloud --username ${apple_id}" in ConfigurePassword(), it became /root/.configwhen I use a non root user and group.

config_dir=/config is set in the Dockerfile build instructions so this should always be present as a system-wide environment variable.

Saying that, I do think there was an issue with the run_as command. It was preserving the environment when switching user, so that would likely result in permissions errors, as stuff like $HOME would still point to the root home directory.

Although, this doesn't explain why I don't see this issue in testing, so there may still be something else not behaving itself.

@boredazfcuk
Copy link
Owner

Closing this off as moving to a non root user should work.

This user also confirms it works using Orbstack: #391

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@boredazfcuk @s-leeq