Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to purge "error" entries from the cache #1320

Open
pierrre opened this issue Aug 3, 2024 · 0 comments
Open

Add an option to purge "error" entries from the cache #1320

pierrre opened this issue Aug 3, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@pierrre
Copy link

pierrre commented Aug 3, 2024

I'm using czkawka/krokiet to find similar/duplicate videos on my computer.
I've got tens of thousands of videos, and tens of TB.
Doing a scan takes days.

When krokiet (actually ffmpeg) is processing a high resolution video (usually 4k), it uses a lot of memory.
If it's configured to use a high number of threads, it may run out of memory. (I guess ffmepg crashes)
When it happens, we get a strange error message Failed to hash file, reason Too short.
Then the file is added to the cache with the error message, and is never scanned again ever.

I found a workaround in this comment #605 (comment)
I just need to reduce the number of threads.
Then, fewer ffmpeg processes are running concurrently, and there is no "out of memory" issue.

The problem: some files are still stored in the cache with the error message.
I can't scan them again with a lower number of threads.
If I want to be able to scan them, I need to delete the entire cache files.
Then it will take me again days to scan all the videos.

Could you add an option/button, to purge the "error" entries from the cache ?
Thank you 😃

@pierrre pierrre added the enhancement New feature or request label Aug 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant