Requesting ability to scan downloaded works for deleted/no longer avaliable stories

[Masterof2D]

Hi, first of all I just want to say this software is fantastic and has been super helpful for me! I know the software supports scanning your current downloads for the "downloading latest version of incomplete fics" feature, so I was wondering if it would be possible to add a way to check each downloaded fics link to see if it's still valid and give a list of the downloaded stories that no longer appear on AO3 (Hidden and/or deleted). An alternate method could be if there is a way to list currently available fics in a category and compare the links with downloaded fics to see what stories are in the downloaded list but not in the current AO3 list. No problem if this is beyond the scope of the project or would be too much to implement, but it would be an extremely helpful resource for me since I download all stories for a list of fandoms, so my download count is over 50,000 and it can be hard to find which in my collection are no longer up. Thanks so much for making this program and your response in advance!

[nianeyna]

You actually can do this with the current version of the script it's just not the most optimized process. Basically what you do is use the "redownload" function, then check the log file for errors after it's complete. Any deleted or hidden or otherwise unavailable fics will show up as an error.

Note the log file has pretty sophisticated filters so it should be relatively easy to filter down to only the failures that happened during your redownload - it's not like you have to individually look through every single log entry, or anything. Although there's a possibility you might have to cross-reference on the work url to get the file names of the fics that were deleted - I can't remember how I built that off the top of my head. All the information is definitely in there though.

The main problem with this is that there's not currently any way to have an empty list of download file types, so it will actually download all of the non deleted fics in at least one file format (therefore it takes at least twice as long as it really needs to, and you have to clean up the files after). So, yeah, not optimized, but doable.

I actually have an issue open to fix this so it doesn't do all the extra downloads, I just haven't gotten around to it yet. You can track #69 to get notified if/when I get that done.