Iris is slow on browsing mopidy-dleyna content, cause it to throw some exceptions and wrongly assign current folder name

[mczerski]

I'm comparing behaviour of the Iris and musicbox_webclient during browsing mopidy-dleyna library content. I have quite large collection of artists and albums on my NAS that exposes its content trough DLNA.
The structure of the library is like this (the folder level is denotet as indentation):

• MUSIC -
  • Albums
    • Some album
      • some tracks
  • all tracks
  • Artists
  • etc. basically some aagregation types
• PHOTOS - this does not matter, i'm not browsing that
• VIDEOs - this does not matter, i'm not browsing that

The result of comparison is that musicbox_webclient is blasing fast and does not make any issues, while Iris is very slow and causes some exceptions rasing in mopiyd-dleyna backend (this is propably issue in mopidy-dleyna itself).

I put some debug logs into mopidy-dleyna to see what kind of LibraryProvider actions are triggered while stepping trough the folders.
For example entering the first level of folders (the one with MUSIC, PHOTOS, VIDEOS) I see that Iris is sending theese requests:
mopidy.backend.LibraryProvider.lookup
mopidy.backend.LibraryProvider.browse
mopidy.backend.LibraryProvider.get_images

While musicbox_webclient only:
mopidy.backend.LibraryProvider.browse

in the case of Iris, mopidy.backend.LibraryProvider.lookup takes 0.5s which is 20x longer that mopidy.backend.LibraryProvider.browse and it makes mopidy-dleyna to fetch list of metadata of all the music content on the NAS (i think).
And the name of the current folder (the name at the bottom) is set to the name of the album of the first item in the fetched list, which is not what the current folder is (Should be the name of the dlna server ?)

mopidy.backend.LibraryProvider.get_images is taking more than 10s but it seems that it is done in the background so it is not the issue. But calling this request at the root of mopidy-dleyna browsing library (where the list of dlna servers should be) causes mopidy-dleyna to raise an exception:

dbus.exceptions.DBusException: com.intel.dleyna.OperationFailed: Unable to parse results of search: Empty 'DIDL-Lite' node in the DIDL-Lite XML:
<DIDL-Lite xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:upnp="urn:schemas-upnp-org:metadata-1-0/upnp/" xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" xmlns:arib="urn:schemas-arib-or-jp:elements-1-0/" xmlns:dtcp="urn:schemas-dtcp-com:metadata-1-0/" xmlns:pv="http://www.pv.com/pvns/" xmlns="urn:schemas-upnp-org:metadata-1-0/DIDL-Lite/"></DIDL-Lite>

In contrast, musicbox_webclient is calling only browse while entering the folder of folders, and when entering the folder of tracks (album) it calls lookup for all files in the folder (one lookup for single file), and get_images for the album.

It seems like fetching metadata of all the audio tracks for every folder level is not necesary and is causing noticable delay. also calling get_images for resources that are not albums also seems not necesarry. At least for my nas server fetching images is very long > 10s.

The other issue with get_images is that I think that to enter next folder level, Iris is waiting for the current level to finish fetching images. So I cannot step trough folders quickly. For example when I go into the Albums folder where there are all of the albums I have, then fetching all the albums images takes very very long and Iris frontend is rising timeout errors:

app.min.js?v=1605864990:116 Uncaught (in promise) Error: Request timed out
    at app.min.js?v=1605864990:116
    at r (app.min.js?v=1605864990:116)

To Reproduce

• install iris and mopidy-dleyna
• reparse some dlna media server with audio tracks
• click on the Browse -> Digital Media Servers -> your server
• start browsing the folders

Expected behavior
stepping trough the mopidy-dleyna library folders being fast and not crashing the browsing feature

Environment details:

• OS: Ubuntu 18.04 for the client, armbian buster for the server
• Browser chrome
• Iris version 3.54.2

[mczerski]

if I change the get_images method of the mopidy-dleyna LibraryProvider to:

def get_images(self, uris):
    return {uri: [] for uri in uris}

then the isris is usable to browse and play music from my dlna server (of course no album covers visible). But still have an issue with current folder name and is a bit slower (0.5 sec) than the musicbox_webclient.

[kingosticks]

s/PlaylistsProvider/LibraryProvider/ ?

[mczerski]

no, LibraryProvider is responsible for browsing feature.

[mczerski]

OMG YES !!!! :D

[mczerski]

Hmm, I think that with get_images there is also another issue, Iris is requesting images for folders (albums) and for for tracks it does not. With musicbox_weblient it is another way around, it requests for images from tracks (actualy only the first track in te album and sets it as the album cover).
It looks that even for mopidy-spotify LibraryProvider Iris is not displaying images whereas musicbox_weblient does.

[jaedb]

Good spotting. The thing to bear in mind that Iris is a feature-rich interface. There will be times that it might be slower than simple frontends.

This touches on something that I've been trying to wrap my brain around for some time. Why bother getting metadata for items (ie Albums, Playlists, etc) that aren't being displayed on screen? Also, when displaying a huge list (ie Playlist with 5,000 tracks) the interface renders them all, making lower-powered computers struggle. But how does this work with remembering scroll position?

My ideal plan is to have grid and list items render only what is being displayed (like a React Native's Flatlist). This could be used to trigger artwork lookups as needed, and only for the displayed items. There are plugins around, but none that are a perfect fit, so it will be a large project.

I think this is the way to approach fixing the issue you've raised, especially given get_images appears to be the heaviest query. Ultimately I feel the issue you've raised is a consequence of the above not existing (yet). I don't want to create a 'fix' specific to dLeyna as it creates code debit and I avoid creating backend-specific functionality like the plague.

[jaedb]

I've been exploring some options, but the issue I keep coming back to is that Mopidy-Dleyna is just slow! The next release breaks down the sequence of lookups/browsing and attempts to reduce unnecessary calls, but there is only so much patching a frontend can do to resolve the backend performance.

FYI: the biggest issue with Mopidy-Dleyna is that library.lookup on any of its URIs returns nothing (which, in my opinion is a bug or a development oversight). As a result, Iris must use the more expensive library.browse to fetch data.

[kingosticks]

the biggest issue with Mopidy-Dleyna is that library.lookup on any of its URIs returns nothing

I don't use Mopidy-Dleyna but having just installed it I don't see this. Do I misunderstand or could it be dependent on what your DLNA server implements? I'm using minidlna here:

HttpServer DEBUG    2021-02-15 17:35:36,919 Received WebSocket message from 127.0.0.1: '{"method":"core.library.lookup","params":{"uris":["dleyna://uuid:4d696e69-444c-164e-9d41-3052cb81d1a3"]},"jsonrpc":"2.0","id":124}'
dLeynaBackend-10 DEBUG    2021-02-15 17:35:36,921 Calling D-Bus method GetAll('',)
MainThread DEBUG    2021-02-15 17:35:36,927 GetAll reply after 0.006s
dLeynaBackend-10 DEBUG    2021-02-15 17:35:36,930 Calling D-Bus method SearchObjectsEx('Type = "music" or Type = "audio"', dbus.UInt32(0), dbus.UInt32(0), ['Album', 'AlbumArtURL', 'Artist', 'Artists', 'Bitrate', 'Date', 'DisplayName', 'Duration', 'Genre', 'TrackNumber', 'Type', 'TypeEx', 'Path', 'RefPath'], '')
MainThread DEBUG    2021-02-15 17:35:36,940 SearchObjectsEx reply after 0.008s
HttpServer DEBUG    2021-02-15 17:35:36,943 Sent WebSocket message to 127.0.0.1: '{"jsonrpc": "2.0", "id": 124, "result": {"dleyna://uuid:4d696e69-444c-164e-9d41-3052cb81d1a3": [
  {"__model__": "Track", "uri": "dleyna://uuid%3a4d696e69-444c-164e-9d41-3052cb81d1a3/363424302430", "name": "A", "artists": [{"__model__": "Artist", "name": "Test-Artist"}], "album": {"__model__": "Album", "name": "Test-Album"}, "bitrate": 1280},
  {"__model__": "Track", "uri": "dleyna://uuid%3a4d696e69-444c-164e-9d41-3052cb81d1a3/363424302431", "name": "B", "artists": [{"__model__": "Artist", "name": "Test-Artist"}], "album": {"__model__": "Album", "name": "Test-Album"}, "bitrate": 1280},
  {"__model__": "Track", "uri": "dleyna://uuid%3a4d696e69-444c-164e-9d41-3052cb81d1a3/3634243245", "name": "Mince 107: Eyebrow Situation", "length": 2531000, "bitrate": 1024000},
  {"__model__": "Track", "uri": "dleyna://uuid%3a4d696e69-444c-164e-9d41-3052cb81d1a3/3634243246", "name": "Mexican Moon", "artists": [{"__model__": "Artist", "name": "Chris Norton; Frank Mizen"}], "album": {"__model__": "Album", "name": "ANW1490 - Country, Blues, Americana"}, "genre": "Romantic", "track_no": 7, "length": 1890000, "bitrate": 1024000},
  {"__model__": "Track", "uri": "dleyna://uuid%3a4d696e69-444c-164e-9d41-3052cb81d1a3/3634243330", "name": "Parsnips 15: Bristle Situation", "length": 2267000, "bitrate": 1024000},
  {"__model__": "Track", "uri": "dleyna://uuid%3a4d696e69-444c-164e-9d41-3052cb81d1a3/3634243331", "name": "The Hodgson Collection Vol. 1", "length": 2111000, "bitrate": 1024000}
]}}'

EDIT: It looks like it needs to use a DLNA search for looking up non-track objects. I did a quick web search and it seems not every DLNA server supports "remote search".

[jaedb]

Hmm I hadn't considered that. I am using the built-in DLNA server that comes with Synology NAS. I guess this is just another variable that makes testing difficult.

[jaedb]

@mczerski how is the latest release (3.57) working for you? Iris now prefers library.lookup but falls back to library.browse, rather than performing both. As a result I expect performance for your use case should have improved.

[mczerski]

@jaedb sorry but I switched to local backend + samba because it works much better. If I find some time I will test new iris against mopidy dleyna and report back.

[jaedb]

No worries. I'll mark this as resolved in the meantime. Please re-open with your findings if the issue persists.

[mczerski]

Hi @jaedb . I tested a little bit new Iris release (3.57). The current behaviour is that after going into: browse -> Digital media servers -> my server name, all tracks of the media server are displayed in a flat list. Is this by the design ? The previous begaviour was to display folder structure defined by the media server (the one I described in the first post in this issue). Is this by the design ? Currently it works fast and reliably but having all tracks of the media server in one flat list may be not very usable.