Large Frequent Library Updates

[brando56894]

Describe the bug
Upon launching Kodi, after Jellyfin logs in, there are a large amount of updates (hundreds) occurring when maybe only a few new files were added.

Please see here for my detailed report and other user reports: https://www.reddit.com/r/jellyfin/comments/ms4ufq/jellyfin_for_kodi_always_has_a_huge_number_of/

To Reproduce
Not exactly sure, but I believe it's related to the TMBD Box Sets plugin. I had it happening almost daily, then while waiting for @mcarlton00 to build the debug plugin, I found that apparently TMDB Box Sets has an issue where if you add something to a collection and then change the name/location of the file the plugin keeps looking for it at the old location (bug report). I attempted to fix that just by removing the TMDB Box Sets plugin but the "bad" records were still in the database. I deleted just my movies library and rescanned them in, without the Box Sets plugin enabled. I had zero or very few updates during the next few days. I reinstalled it and rescanned the collections, over the next few days things were being updated, and the large batches of updates started again. It was a lot lower than before, only a hundred or so, but then yesterday it popped up for 700+ notifications when only one new item had been added that day.

Expected behavior
There are no large daily updates when nothing has changed.

Logs
I can't post the text anywhere because it's about 186,000 lines for about 4 hours worth of logs, the file is 17 MB, so I hosted it on my Google Drive: https://drive.google.com/file/d/1rrmqmvW55GenbDq9WXu3itIKoMLf2nTv/view?usp=sharing

System (please complete the following information):

• OS: Arch Linux (Jellyfin Server)/Nvidia Shield Android TV v8.2.2 (Kodi client)
• Jellyfin Version: 10.7.2
• Kodi Version: 19.0.0
• Addon Version: 0.7.1+py3
• Playback Mode: Native

[mcarlton00]

This seems to confirm my theory that the server is giving us unnecessary updates, but some thoughts on your logs.

---

First off, debug logging in this case doesn't particularly help, unfortunately. It gives us the full metadata for each object, but since we don't know what that metadata was before these most recent changes it doesn't give us a whole lot of insight to what's happened. That can significantly cut down on your log size if we need to collect more.

---

Second, it looks like 27 of the updates you received were "valid". Maybe not necessarily wanted, given how they changed, but the changes we received from the server do explain why Kodi updated them. You can see these with

% grep -A 2 column kodi.log | sed -En "s/(.*)movies.py:6[0-9]//p"

A lot of them seem to be getting switched from having a value to None. I can't imagine why that's happened, but it makes sense why we'd be getting change requests for that.

---

Now the more interesting part, and what seems to indicate my theory is correct, is the number of updates we received that had no changes in the database.

% grep 'no changes' kodi.log | wc -l
754

Looks to be a lot of American Dad, Family Guy, and Whose Line in the list. You can see episode titles with

% grep 'no changes' kodi.log | sed -En "s/^(.*).py:[0-9]+ //p"

So this seems to mean is that the kodi sync queue plugin is marking things as updated even if we don't necessarily care about the piece of data that's been updated. We were discussing this in chat the other day and started some thoughts on it here jellyfin/jellyfin-plugin-kodisyncqueue#44

---

Overall, it sounds like there's nothing we can do about it here and I'll need to go beg the server/plugin folks to poke around there, but it's good reference material to influence future API decisions. The bit about TMDB Boxsets having an effect here is very odd, but good information to have and another avenue to look into.

[brando56894]

Interesting! Glad you were able to glean some information from that. I wasn't sure if debug logs were going to be helpful since you modded the plugin yourself to get more info, I'll disable it now in case it happens again.

I'm going to mass rename the files/folders my movie library to make them all homogenous and then I can see if that causes huge daily updates again, because I haven't really had any since I posted this.

[razordullwit]

I'm encountering a similar issue. Almost every time I open Kodi on my Shield TV, it wants to sync my entire library, even if I've only updated a few things. Expected behavior would be that it only syncs new or actually changed items.

The problem is definitely upstream, but I can't figure out what is triggering the updates. I can't figure out if it's a scheduled task or a plugin or what. It's not logged anywhere, and I don't know how to access the database (nor would I necessarily want to).

Has anyone found the primary culprit for problems like this yet? Googling has not helped.