Upload not uploading files that are deleted on the server.

[vangorra]

Reproduction:

• Get a google takeout zip.
• Extract to /externalLibrary2
• Setup an external path as /externalLibrary2
• Let immich do it's processing.
• Remove /externalLibrary2 from the external library paths.
• Tell immich to scan libraries.
• Tell immich to refresh the library jobs.
• Wait for the jobs to complete.
• Use immich-go to import the original zip as a google takeout.

Outcome:

• No files are ever uploaded.
  Log is filled with this line for all images:

 INFO | server has same asset | file="Takeout/Google Photos/05/IMG_9802-1.JPG" reason="An asset with the same name:\"IMG_9802-1.JPG\", date:\"2009-04-05 22:19:30\" and size:1.0 MB exists on the server. No need to upload." time="2025-01-07T03:50.13 PST"
 INFO | added to an album | file="Takeout/Google Photos/05/IMG_9802-1.JPG" album=05 time="2025-01-07T03:50.13 PST"

Even though the immich db shows it was deleted:

                  id                  | deviceAssetId |               ownerId                |    deviceId    | type  |               originalPath               |     fileCreatedAt      |     fileModifiedAt     | isFavorite | duration | encodedVideoPath |                  checksum                  | isVisible | livePhotoVideoId |           updatedAt           |           createdAt           | isArchived | originalFileName | sidecarPath |                  thumbhash                   | isOffline |              libraryId               | isExternal |         deletedAt          |     localDateTime      | stackId |             duplicateId              | status 
--------------------------------------+---------------+--------------------------------------+----------------+-------+------------------------------------------+------------------------+------------------------+------------+----------+------------------+--------------------------------------------+-----------+------------------+-------------------------------+-------------------------------+------------+------------------+-------------+----------------------------------------------+-----------+--------------------------------------+------------+----------------------------+------------------------+---------+--------------------------------------+--------
 5531589f-c497-44f8-9719-928d052b5e8c | IMG_9774.JPG  | 8c8d74a3-33ed-4c5a-a45e-d1266691f433 | Library Import | IMAGE | /externalLibrary2/Me/05/IMG_9774.JPG | 2009-04-06 05:15:37+00 | 2025-01-05 17:53:35+00 | f          |          |                  | \x4a9d35fa613f19530ccd03651e9635b5b4a1ffa4 | t         |                  | 2025-01-07 22:52:25.877293+00 | 2025-01-07 01:45:43.773083+00 | f          | IMG_9774.JPG     |             | \xddf8092d827339775f88656e78b798867a8087261f | t         | 9a630d82-22e1-48fc-b8a9-ff6b2f3bd389 | t          | 2025-01-07 22:52:25.876+00 | 2009-04-05 22:15:37+00 |         | 6c684911-9573-43b0-af1b-36550a92fc7d | active
(1 row)

Expected result:

• immich-go should determine if the file exists before avoiding upload.

[vangorra]

Older versions of Immich had a feature to remove the offline assets. It doesn't appear to be in the latest versions. To immediately mitigate this issue, I ran the following queries to remove the offline assets and related data.

Disclaimer: You should not do this unless you know what you are doing!

DELETE FROM albums_assets_assets WHERE "assetsId" IN (SELECT id FROM assets WHERE "isOffline"='t');
DELETE FROM asset_faces WHERE "assetId" IN (SELECT id FROM assets WHERE "isOffline"='t');
DELETE FROM asset_files WHERE "assetId" IN (SELECT id FROM assets WHERE "isOffline"='t');
DELETE FROM asset_stack WHERE "primaryAssetId" IN (SELECT id FROM assets WHERE "isOffline"='t');
DELETE FROM memories_assets_assets WHERE "assetsId" IN (SELECT id FROM assets WHERE "isOffline"='t');
DELETE FROM shared_link__asset WHERE "assetsId" IN (SELECT id FROM assets WHERE "isOffline"='t');
DELETE FROM tag_asset WHERE "assetsId" IN (SELECT id FROM assets WHERE "isOffline"='t');

DELETE FROM assets WHERE "isOffline"='t';

[simulot]

immich-go checks the list of photos available on the server before uploading. Apparently, immich still have a reference to them after having de-referenced the external library.

I think there is a way to force immich to re-scan the library, and remove off line assets

I'm not familiar with extetnal libraries. I think that the immich API indicates where the asset is stored, and I can exclude external ones from the duplication check.