error on non-existing image

[haayman]

My backup never succeeds anymore, because it tries to download this image photos/2022/07/20220708_135903.jpg which, as far as I can see, doesn't exist. So apparently the search returns a result that doesn't exist?
Anyway, it would be nice if this timeout wouldn't result in stopping alltogether

05-29 17:01:32 gphotos_sync.Main INFO version: 0.1.dev1+g208a216, database schema version 5.7
05-29 17:01:32 gphotos_sync.BadIds DEBUG bad_ids file, loaded 44 bad ids
05-29 17:01:32 gphotos_sync.GooglePhotosIndex WARNING Indexing Google Photos Files ...
05-29 17:01:32 gphotos_sync.GooglePhotosIndex INFO searching for media start=2024-05-28 10:53:32, end=None, videos=True
05-29 17:01:32 gphotos_sync.GooglePhotosIndex DEBUG mediaItems.search with body:
{'pageToken': None, 'pageSize': 100, 'filters': {'dateFilter': {'ranges': [{'startDate': {'year': 2024, 'month': 5, 'day': 28}, 'endDate': {'year': 3000, 'month': 1, 'day': 1}}]}, 'mediaTypeFilter': {'mediaTypes': ['ALL_MEDIA']}, 'featureFilter': {'includedFeatures': ['NONE']}, 'includeArchivedMedia': False}}
05-29 17:01:33 gphotos_sync.GooglePhotosIndex WARNING indexed 0 items
05-29 17:01:33 gphotos_sync.GooglePhotosDownload WARNING Downloading Photos ...
05-29 17:01:33 gphotos_sync.GooglePhotosDownload INFO downloading 1 photos/2022/07/20220708_135903.jpg
05-29 17:02:33 gphotos_sync.LocalData INFO Saving Database ...
05-29 17:02:33 gphotos_sync.LocalData INFO Database Saved.
05-29 17:02:33 gphotos_sync.Main ERROR
Process failed.
Traceback (most recent call last):
File "/root/.local/lib/python3.10/site-packages/gphotos_sync/Main.py", line 493, in main
self.start(args)
File "/root/.local/lib/python3.10/site-packages/gphotos_sync/Main.py", line 433, in start
self.do_sync(args)
File "/root/.local/lib/python3.10/site-packages/gphotos_sync/Main.py", line 401, in do_sync
files_downloaded = self.google_photos_down.download_photo_media()
File "/root/.local/lib/python3.10/site-packages/gphotos_sync/GooglePhotosDownload.py", line 178, in download_photo_media
self.do_download_complete(futures_left)
File "/root/.local/lib/python3.10/site-packages/gphotos_sync/GooglePhotosDownload.py", line 326, in do_download_complete
e = future.exception(timeout=timeout)
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 496, in exception
raise TimeoutError()
concurrent.futures._base.TimeoutError
05-29 17:02:33 gphotos_sync.Main WARNING Done.

[haayman]

never mind. I threw away the .sqlite and now everything works nicely

[haayman]

after running it twice successfully I'm now stuck again on the same image

[gilesknap]

@haayman that's annoying. I used to have a BadIds file which recorded the files that fail to download and then skips them until you delete the badids text file at which point it tries again.

I think that is still in operation but perhaps it does not get used for files that time out.

That might not be hard to fix so I'll take a look.

[gilesknap]

OK I've added timeout error to the bad_ids and also made sure that the thread is cleaned up on unexpected errors.

Can't easily test your case but I'll get a release out this weekend so you can try it.

[gilesknap]

Please try out 3.2.5 when it is released and see if this fixes your issue.

[haayman]

07-14 08:42:37 WARNING gphotos-sync 3.2.5 2024-07-14 08:42:37.071143
07-14 08:42:37 WARNING Indexing Google Photos Files ...
07-14 08:42:38 WARNING indexed 7 items
07-14 08:42:38 WARNING Downloading Photos ...
07-14 08:43:38 ERROR
Process failed.
Traceback (most recent call last):
File "/root/.local/lib/python3.12/site-packages/gphotos_sync/main.py", line 553, in main
self.start(args)
File "/root/.local/lib/python3.12/site-packages/gphotos_sync/main.py", line 471, in start
self.do_sync(args)
File "/root/.local/lib/python3.12/site-packages/gphotos_sync/main.py", line 439, in do_sync
files_downloaded = self.google_photos_down.download_photo_media()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/gphotos_sync/GooglePhotosDownload.py", line 178, in download_photo_media
self.do_download_complete(futures_left)
File "/root/.local/lib/python3.12/site-packages/gphotos_sync/GooglePhotosDownload.py", line 332, in do_download_complete
e = future.exception(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 494, in exception
raise TimeoutError()
TimeoutError
07-14 08:43:38 WARNING Done.

What's weird is that the downloads seem to work alright. I'm seeing yesterday's photos on my drive. So I don't know what's going wrong here