Very slow on mounted volume on Mac (due to SQLite)

[zzolo]

First let me just say that beets is amazing. I just started using it a few days ago and am totally in love.

Problem

Not sure this is really specific to beets, but running beets with a mounted volume is extremely slow on a Mac, due to SQLite slowness. I have my music library, database, and config on a NAS that gets mounted to my Mac and am running the beet command locally on the Mac. It is extremely slower than if the library is local.

$ time beet -c /Volumes/Multimedia/tunes/beets.conf -vv import /Volumes/Multimedia/tunes-old/albums/W/Wilco/Summer\ Teeth/
[...]
real    7m35.727s

$ time beet -l ./test-beets.db -d ./test-beets-lib/ -vv import /Volumes/Multimedia/tunes-old/albums/W/Wilco/Summer\ Teeth/ -I
[...]
real    0m47.539s

So, I am wondering if anyone else has run into this (I don't think this is a really weird situation)? And if so, is there a work around that doesn't involve storing my beets database locally. And if not, there is some threads on the internet suggesting that some SQLite config might be able to get things going a bit fast, so any pointers on where I might try to put those for a pull request would be helpful.

Thanks!

Setup

• OS: Mac OSX 10.11.4 and QNAP NAS
• Python version: 2.7.10
• beets version: 1.3.17
• Turning off plugins made problem go away (yes/no): no

My configuration (output of beet config) is:

As you can see, there are plugins installed that fetch other stuff, but that would not explain an almost 10x increase in speed.

directory: /Volumes/Multimedia/tunes/
library: /Volumes/Multimedia/tunes/beets.db

import:
  move: no
  incremental: yes

paths:
  default: albums/%upper{%left{$artist_ranked,1}}/$artist_ranked/$album%aunique{}/$track - %if{$different_artist,$artist - }$title
  singleton: loose/$artist - $title
  comp: compilations/$album%aunique{}/$track - $artist - $title
  albumtype:soundtrack: soundtracks/$album%aunique{}/$track - $artist - $title

plugins: acousticbrainz fetchart lyrics lastgenre inline embedart

item_fields:
  artist_ranked: albumartist_sort if albumartist_sort else albumartist if albumartist else artist
  different_artist: True if albumartist != artist and not comp else False

embedart:
  maxwidth: 800

[zzolo]

Here is my current hack/workaround, which may be the best option. Basically I create a function/alias that has the database locally and then copies the relevant files to the mounted volume. I can also add a check to make sure the volume is mounted.

function mybeet() {
  if [ -e /Volumes/Multimedia/tunes/ ]; then
    beet "$@" && \
    cp ~/.config/beets/config.yaml /Volumes/Multimedia/tunes/beets.conf && \
    cp ~/.config/beets/library.db /Volumes/Multimedia/tunes/beets.db && \
    cp ~/.config/beets/beets.log /Volumes/Multimedia/tunes/beets.log;
  else
    echo "Music library not found."
  fi
}

This goes in my .bash_profile. And I can do the same beet command just as mybeet instead.

$ mybeet import /music/

Much, much faster.

[jackwilsdon]

It seems like #1710 could help your situation but it doesn't look like anyone is actively working on it.

[sampsyo]

Interesting! We do actually recommend, normally, that the database not go on a network filesystem. Performance is one reason, but there are also weird issues when multiple clients try to access the same database at the same time. Your shell-script solution that copies the local database to the remote is very clever!

I don't have any great advice right now, but for a long-term solution, I prefer #718 over #1710. That is, we'd eventually like to develop a beets API that lets you run all your beet commands locally against a database hosted on a server.

[zzolo]

Thanks for the info. I am not that interested in running a server-level database for my music collection. I like the portability of SQLlite/file.

I am not sure there is that much to do. I read some options to use specific PRAGMA options for SQLite but there is mixed info about how effective it is. I'll close this for now and maybe someone can use the bash function above if they run into similar situations.