[Feature Request] Add the ability to remove duplicates in a playlist

[CatKasha]

I think it would be a useful feature for those who use their own playlists

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[illustribe]

+1 for this one!

[ChiefMikeK]

Already has been implemented

1. Open the play list then long press on the duplicate item.
2. press Delete
   
   

[illustribe]

And do this one by one? On a +300 videos list??
Maybe the requested feature isn't well explained, or I'm on the wrong thread...
There should - or would be great to - be an option to either disallow adding twice the same video to a list, or to batch remove all duplicates.
Cheers and thanks you anyway :)

[CatKasha]

@illustribe this is exactly what i wanted to answer to ChiefMikeK 😃
and way to implement this feature too!

[opusforlife2]

Closing in favour of #4186.

[Jared234]

I can start working on this.
However, before I start the implementation, I have a question regarding the deletion process.
IMO there are 2 ways you can delete duplicates:

1. You get all streams of a playlist (with duplicates already removed) via a database query. After clearing the database completely, you simply re-add those streams.
2. The other option is to find the duplicates and remove them in the same query. However, after that we would have to rebuild the corresponding index.

I think that the first approach is the most efficient way to handle the deletion of streams from the playlist. Currently, when we delete streams from the playlist (Remove Watched or Delete), we only make local changes to the itemListAdapter. Before exiting the playlist view, these changes are then persisted using the saveImmediate function in the LocalPlaylistFragment. This means that we only need to update the database once, rather than after each individual modification.

However, I am open to suggestions. Do you have an opinion on which approach I should choose?

[tsilvs]

Do you have an opinion on which approach I should choose?

@Jared234 which option will work faster on playlists that could contain ~100 videos w/ ~4 of them being duplicates of one video and ~3 - of the other?

[Jared234]

Unfortunately, I am not quite sure which one would be the most performant or best overall method for deleting duplicates. That is also the reason why I initially created the comment. However, at least from the user's perspective, I believe that option 1 will most likely be faster.
Are there any other developers who want to share their opinion on this?

[SameenAhnaf]

What about long press on any playlist for bulk selection? Selecting only greyed out playlist will show all three icons Remove, Deduplicate and +New Entry. If any of the selected playlists was not greyed out, only +New Entry would have been shown.

[Jared234]

I agree that bulk selection could be a great feature. However, I don't think it fits within the scope of this issue, as it wouldn't be a small addition to it, but a complete change to the user interface.

[opusforlife2]

We could always decide that we want to fulfill this FR via bulk selection (in combo with your duplicates warning PR), in which case this would remain open until someone implements that.

Personally, that is also the solution I would go for. Now that users are warned about duplicates, there will be no more unintentional duplicates. Having a button just to remove them will become too overly specific a feature for later releases.

Edit: actually, once the sorting-filtering framework is added, most sorts would group duplicates together. Or there could even be a 'duplicates' filter.