[Latest Edits] How to download portrait image with bokeh effect

[yuuxie]

Summary

[NOTE]: # For portrait photos, it will download the original photo, but not the bokeh one.

For reference, if I import photos from iphone with "Image Capture" app on mac, it will import two photos, one for the original photo, the other with the bokeh effect.

So, from icloud, how can I download the portrait image with bokeh effect?

[AndreyNikiforov]

Portraits are saved on iCloud as "edits". icloudpd downloads original image, not latest/edited, as a result portraits are not captured.

[yuuxie]

so, is there a way to download the "edits"? eg, the AAE file?

[AndreyNikiforov]

No, AAE files are not downloaded in the current version of the icloudpd. If they were, how would you use them? Is any of the photo processing software support "re-applying" AAE to the image?

[yuuxie]

Seems the following change works
the portrait resource is in another CPLAsset record with download url in resJPEGFullRes

diff --git a/src/icloudpd/base.py b/src/icloudpd/base.py
index 2b1a5f3..1aa0dc1 100644
--- a/src/icloudpd/base.py
+++ b/src/icloudpd/base.py
@@ -30,7 +30,7 @@ from icloudpd import download
 from icloudpd.email_notifications import send_2sa_notification
 from icloudpd.string_helpers import truncate_middle
 from icloudpd.autodelete import autodelete_photos
-from icloudpd.paths import clean_filename, local_download_path
+from icloudpd.paths import clean_filename, local_download_path, filename_with_size
 from icloudpd import exif_datetime
 # Must import the constants object so that we can mock values in tests.
 from icloudpd import constants
@@ -242,6 +242,17 @@ CONTEXT_SETTINGS = {"help_option_names": ["-h", "--help"]}
               is_flag=True,
               default=False,
               )
[email protected](
+    "--enable-edited-photos",
+    help="Download the edited photos, such as portrait (default: Not download)",
+    is_flag=False,
+)
[email protected](
+    "--edited-photo-size",
+    help="Edited(like portrait) Photo size to download (default: full)",
+    type=click.Choice(["full", "medium", "thumb"]),
+    default="full",
+)
 # a hacky way to get proper version because automatic detection does not
 # work for some reason
 @click.version_option(version="1.17.3")
@@ -282,7 +293,9 @@ def main(
         delete_after_download: bool,
         domain: str,
         watch_with_interval: Optional[int],
-        dry_run: bool
+        dry_run: bool,
+        enable_edited_photos: bool,
+        edited_photo_size
 ):
     """Download all iCloud photos to a local directory"""
 
@@ -337,6 +350,8 @@ def main(
                     set_exif_datetime,
                     skip_live_photos,
                     live_photo_size,
+                    enable_edited_photos,
+                    edited_photo_size,
                     dry_run) if directory is not None else (
                     lambda _s: lambda _c,
                     _p: False),
@@ -387,6 +402,8 @@ def download_builder(
         set_exif_datetime: bool,
         skip_live_photos: bool,
         live_photo_size: str,
+        enable_edited_photos: bool,
+        edited_photo_size: str,
         dry_run: bool) -> Callable[[PyiCloudService], Callable[[Counter, PhotoAsset], bool]]:
     """factory for downloader"""
     def state_(
@@ -599,6 +616,47 @@ def download_builder(
                                     "Downloaded %s",
                                     truncated_path
                                 )
+
+            # Also download the edited photo (such as portrait) if present
+            if enable_edited_photos:
+                edited_size = edited_photo_size + "Edited"
+                if edited_size in photo.versions:
+                    version = photo.versions[edited_size]
+                    filename = version["filename"]
+                    if edited_photo_size != "full":
+                        filename = f"-{edited_photo_size}.".join(filename.rsplit(".", 1))
+
+                    edited_download_path = os.path.join(download_dir, filename)
+                    edited_file_exist = os.path.isfile(edited_download_path)
+                    if only_print_filenames and not edited_file_exist:
+                        print(edited_download_path)
+                    else:
+                        if edited_file_exist:
+                            edited_file_size = os.stat(edited_download_path).st_size
+                            edited_photo_size_num = version["size"]
+                            if edited_file_size != edited_photo_size_num:
+                                edited_download_path = f"-{edited_photo_size_num}".join(
+                                    edited_download_path.rsplit(".", 1)
+                                )
+                                logger.debug(
+                                    "%s deduplicated",
+                                    truncate_middle(edited_download_path, 96)
+                                )
+                                edited_file_exist = os.path.isfile(edited_download_path)
+                            if edited_file_exist:
+                                logger.debug(
+                                    "%s already exists",
+                                    truncate_middle(edited_download_path, 96)
+                                )
+                        if not edited_file_exist:
+                            truncated_path = truncate_middle(edited_download_path, 96)
+                            logger.debug("Downloading %s...", truncated_path)
+                            download_result = download.download_media(
+                                logger, dry_run, icloud, photo, edited_download_path, edited_size)
+                            success = download_result and success
+                            if download_result:
+                                logger.info("Downloaded %s", truncated_path)
+
             return success
         return download_photo_
     return state_
diff --git a/src/pyicloud_ipd/services/photos.py b/src/pyicloud_ipd/services/photos.py
index a6dfe58..8dd43ae 100644
--- a/src/pyicloud_ipd/services/photos.py
+++ b/src/pyicloud_ipd/services/photos.py
@@ -544,6 +544,12 @@ class PhotoAsset(object):
         u"thumb": u"resVidSmall"
     }
 
+    EDITED_VERSION_LOOKUP = {
+        u"fullEdited": u"resJPEGFull",
+        u"mediumEdited": u"resJPEGMed",
+        u"thumbEdited": u"resJPEGThumb",
+    }
+
     @property
     def id(self):
         return self._master_record['recordName']
@@ -667,6 +673,57 @@ class PhotoAsset(object):
 
                     self._versions[key] = version
 
+            # Resource for the edited photos (portrait)
+            # The portrait photo resources are in the
+            if self.item_type != "movie":
+                for key, prefix in self.EDITED_VERSION_LOOKUP.items():
+                    if '%sRes' % prefix in self._asset_record['fields']:
+                        f = self._asset_record['fields']
+
+                        type_entry = f.get('%sFileType' % prefix)
+
+                        # Follow the convention of "Image Capture" that add E to the beginning.
+                        edited_filename = re.sub(r'_(\d{4})\.', r'_E\1.', self.filename)
+                        if edited_filename == self.filename:
+                            # In case the file name not match the above pattern
+                            edited_filename = f"-EDITED.".join(self.filename.rsplit(".", 1))
+
+                        # In case the file type extension is not the same as original file.
+                        if type_entry and type_entry['value'] in self.ITEM_TYPE_EXTENSIONS:
+                            file_extension = self.ITEM_TYPE_EXTENSIONS[type_entry['value']]
+                            edited_filename_splits = edited_filename.rsplit(".", 1)
+                            if edited_filename_splits[1] != file_extension:
+                                edited_filename = edited_filename_splits[0] + "." + file_extension
+
+                        version = {'filename': edited_filename}
+
+                        if type_entry:
+                            version['type'] = type_entry['value']
+                        else:
+                            version['type'] = None
+
+                        width_entry = f.get('%sWidth' % prefix)
+                        if width_entry:
+                            version['width'] = width_entry['value']
+                        else:
+                            version['width'] = None
+
+                        height_entry = f.get('%sHeight' % prefix)
+                        if height_entry:
+                            version['height'] = height_entry['value']
+                        else:
+                            version['height'] = None
+
+                        size_entry = f.get('%sRes' % prefix)
+                        if size_entry:
+                            version['size'] = size_entry['value']['size']
+                            version['url'] = size_entry['value']['downloadURL']
+                        else:
+                            version['size'] = None
+                            version['url'] = None
+
+                        self._versions[key] = version
+
         return self._versions
 
     def download(self, version='original', **kwargs):

[AndreyNikiforov]

@yuuxie pls submit PR with the change. I suspect we may need some flag to allow original and/or latest (or may be download both by default? -- would love to hear your thoughts). We'll need tests for new behavior as well. Thanks for improving icloudpd!

[yuuxie]

#773 .
This is the PR.
it has a flag --enable-edited-photos to enable the function.
And it could download the default original one, and the portrait one together

[ja-albert]

I tested the download of edited photos with the PR and so far, it does look good and does what I would expect. I compared the downloaded images with a direct download from iCloud and a direct copy from my iPhone: None of the alternatives gave me photos/videos that were not downloadable with the PR.

However, I have a few suggestions:

• Rename the new flage to --skip-edited-photos (Default: Download edited photos), to be consistent with the wording of the similar options --skip-videos and --skip-live-photos.
• Move both new flags (enable download and image size) up to their similar options in the help message - that is, next to --skip-xxx and --xxx-size respectively`. As above, this should make the new flags more consistent with the rest of the application.

And of course a big thank you for this PR and icloudpd in general - with them, I can completely avoid other tools :)

[AndreyNikiforov]

1.19.0 can download adjusted files (edits, portraits etc)