[Latest Edits] Portrait photos or blurred backgrounds do not seem to be well preserved and downloaded.

[wclebb]

Summary

Hope to provide photos that retain the portrait elements, rather than original photos without processing.

Context

It is not consistent with the iPhone photo. I hope the iPhone photo is consistent with the download.
Portrait photos or blurred backgrounds do not seem to be well preserved and downloaded.
icloudpd downloaded the most original photos, not processed photos.

I tried two command codes, for example
--size (original|medium|thumb)
--live-photo-size (original|medium|thumb)

It seems to look exactly the same, the file size is the same (but I did not verify the MD5)
--force-size I don't know what command this is or how to operate.

See
iCloud.com Download Photo

Github iCloudpd - Download Photo

————

[menkej]

If I get you right: You edited a picture on your Phone and icloudpd does download the original and not the changed picture. Is this your issue?

By design Apple only saves the original image in iCloud. The changes are just "described" in a meta file and the client software (Photos-App) applies these changes when you open the image. Currently it is not possible to download the edited image as far as I know...

[wclebb]

If I get you right: You edited a picture on your Phone and icloudpd does download the original and not the changed picture. Is this your issue?

By design Apple only saves the original image in iCloud. The changes are just "described" in a meta file and the client software (Photos-App) applies these changes when you open the image. Currently it is not possible to download the edited image as far as I know...

Thank you.
But I not edited Photo.

I use portrait mode to take a picture of my face, and at the same time I get a picture with a blurred background.
I don’t have any edited photos. I just use icloudpd to download photos without portrait mode and blur. It does not seem to use portrait mode photos.

In fact, if you click the Download button on icloud.com, you will download a photo face with a blurred background and portrait.

But if you use icloudpd download, there will be no photos with blurred background and portrait processing.

Since it is 11:30 in the evening on my side, the test cannot be performed.
But it can only be said that iCloud Web Download is indeed a photo with portrait mode and blur;
For some reason, icloudpd can only download photos without portrait mode and blur.

I found an example
Use built-in camera, STAGE LIGHT MONO mode
IMG_0049.HEIC, The same file name
——

Forgive me for not being able to use English and use Google Translate to convey my thoughts.
In fact, I don’t know how icloudpd works, but it seems that you only need to use the iCloud Web Download button to download filter photos, such as STAGE LIGHT MONO filter photos. If icloudpd can use iCloud Web download, it is possible to get the same effect of photos, even jpeg, without conversion. This may be good news for devices and operating systems that are not compatible with the HEIC format.

Because I think it would be better if iPhone photos and downloads have the same effect. However, the download does not have some filter effects, so I am puzzled.

And I almost delete all photos.

[Sebolains]

If I get you right: You edited a picture on your Phone and icloudpd does download the original and not the changed picture. Is this your issue?

By design Apple only saves the original image in iCloud. The changes are just "described" in a meta file and the client software (Photos-App) applies these changes when you open the image. Currently it is not possible to download the edited image as far as I know...

Hi @menkej,

I did some quick checking, and it is possible to get edited photos from iCloud. The photos are downloadable as JPEGs and are in the asset record under the name resJPEGFullRes. Here is a quick script that shows this:

from pyicloud_ipd import PyiCloudService

email = "[email protected]"
password = "password123"

api = PyiCloudService(email, password)
photos = api.photos.all
photos_iter = iter(photos)

photo_noportrait = next(photos_iter, None)
photo_portrait = next(photos_iter, None)

asset_record_field_noportrait = photo_noportrait._asset_record['fields'].keys()
asset_record_field_portrait = photo_portrait._asset_record['fields'].keys()

print(asset_record_field_portrait - asset_record_field_noportrait)

which gives the output:
{'resJPEGThumbRes', 'resJPEGThumbFileType', 'resJPEGMedRes', 'adjustmentType', 'resJPEGFullFingerprint', 'resJPEGThumbWidth', 'resJPEGThumbFingerprint', 'resJPEGThumbHeight', 'resJPEGMedFingerprint', 'resJPEGFullHeight', 'resJPEGMedWidth', 'resJPEGFullRes', 'resJPEGMedFileType', 'resJPEGMedHeight', 'resJPEGFullWidth', 'resJPEGFullFileType'}

It is clear from this that the attribute adjustmentType is indicated that there has been an adjustment (or edit) to the photo, and the adjusted full-resolution image is stored in the resource resJPEGFullRes.

This difference is the same between a non-edited and an edited photo. Since iCloud treats portait mode as any other edit, I think this issue can be solved by simply adding an option to icloudpd that downloads edited versions of the photos as well. (personally I would leave this on by default to avoid people losing data like the OP mentioned).

Implementing this would require changing some pyicloud_pd code, since the versions() property is currently only looking into the master records, and not the asset records. This could be done by changing the versions property code, or by adding an edited property.

Here is a very manual and messy way to add the edited version without having to modify pyicloud_pd. It is not a clean way of doing it as it is essentially duplicating all of pyicloud code, but might serve as a quick guideline as to how to implement it:

photo.has_edits = u"adjustmentType" in photo._asset_record['fields']
if photo.has_edits:
    prefix = u"resJPEGFull"

    f = photo._asset_record['fields']

    filename = photo.filename
    
    # Modifying filename and extension for edit
    filetype_entry = f.get('%sFileType' % prefix)
    if filetype_entry:
        filetype = filetype_entry['value']
    else:
        filetype = u"public.jpeg"

    extension = photo.ITEM_TYPE_EXTENSIONS[filetype]
    filename = "%s.%s" % (filename.split('.')[0] + "E" , extension)
    version = {'filename': filename}

    width_entry = f.get('%sWidth' % prefix)
    if width_entry:
        version['width'] = width_entry['value']
    else:
        version['width'] = None

    height_entry = f.get('%sHeight' % prefix)
    if height_entry:
        version['height'] = height_entry['value']
    else:
        version['height'] = None

    size_entry = f.get('%sRes' % prefix)
    if size_entry:
        version['size'] = size_entry['value']['size']
        version['url'] = size_entry['value']['downloadURL']
    else:
        version['size'] = None
        version['url'] = None

    type_entry = f.get('%sFileType' % prefix)
    if type_entry:
        version['type'] = type_entry['value']
    else:
        version['type'] = None

    photo.versions #make sure photo._versions is populated
    photo._versions['edited'] = version

Note all the necessary work on the filename. Apparently, even though the field is called resJPEGFullRes, iCloud stores some edits in HEIC format. For me, this seems to happen on photos I've edited (or portait photos I've taken) starting in iOS 14, while portrait photos before then have their edits in iCloud as JPEGs. These are the same file formats shown in file explorer when connecting my iPhone directly through USB to my PC.

Sorry if I made any obvious mistakes! I'm not too familiar with the inner workings of pyicloud.

[menkej]

Thanks for your analysis, @Sebolains ! Seems like there is a chance to get this done.

I'm currently deep into trying to port icloudpd back to the original pyicloud (and changing pyicloud to make this possible), and I'm also not familiar with pyicloud at all... ;-) So I like to postpone diving into into your suggestion to afterwards.

If you think you can bring this to icloudpd without changing pyicloud, feel free to submit a pull request and we can integrate it together!

[kristian-mi]

@Sebolains I find it very useful if we could add support for downloading the edited version of the photos as well - have you had any chance writting this up as a patch/make a pull request?

[BeginLjm]

I made a simple modification, the edited photo changed the origin download default to resJPEGFullRes https://github.com/BeginLjm/icloud_photos_downloader

[angadsingh]

I just noticed this today and its super disappointing. I was well into assuming the copy of photos and videos downloaded from icloudpd are as good as a backup. At the very least you should put up all such limitations in clear text in the README

[jcass77]

Upstream picklepete/pyicloud#347 expands on @Sebolains approach, and should ensure that the edited versions of photos are downloaded instead of the originals.

[revpdev]

Hi all,
Based on @Sebolains investigation, I built a working solution to download edited photos and videos. I apologize, I am totally newbie to github so not sure how to properly contribute, so in case it can help, I am attaching the diffs of the 3 files that I updated to make it work: autodelete.py and base.py from src/icloudpd and photos.py from src/pyicloud_ipd/services.
It is basically adding the "edited" size for download that will look for resJPEGFullRes photos in photo._asset_record and resVidFullRes videos in photo._asset_record as well.
I adjusted a few debug logs as well.
changes made on version 1.15.1.

base.py.diff.txt
autodelete.py.diff.txt
photos.py.diff.txt

edit: I just saw the code from picklepete/pyicloud#347 and it is indeed more elegant and complete, though only available in pyicloud.

I also noted as well that using the --set-exif-datetime is breaking the file_size comparison between the icloud file and the downloaded file. indeed, setting the exif date time is adding a few bytes to the file and thus the file is downloaded twice. Maybe worth a new bug ?

[AndreyNikiforov]

1.19.0 can download adjusted files (edits, portraits etc)