Tackle high page-load (and api response) times

[fm3]

slowest requests (avg of last three days):

  32300 ms    /controllers.ReportController.projectProgressOverview
  29000 ms    /controllers.StatisticsController.webKnossos
   5380 ms    /controllers.DataSetController.list
   4560 ms    /controllers.WKDataStoreController.updateAll
   3970 ms    /controllers.StatisticsController.users
   3550 ms    /controllers.UserController.list
   3360 ms    /controllers.DataSetController.thumbnail
   3250 ms    /controllers.ReportController.openTasksOverview
   1750 ms    /controllers.TimeController.getWorkingHoursOfAllUsers
   1460 ms    /controllers.ProjectController.listWithStatus
   1100 ms    /controllers.ProjectController.list
    972 ms    /controllers.AnnotationController.duplicate
    785 ms    /controllers.AnnotationIOController.upload
    457 ms    /controllers.AnnotationIOController.download
    445 ms    /controllers.ProjectController.transferActiveTasks
    397 ms    /controllers.TaskController.createFromFiles
    383 ms    /controllers.TaskController.peekNext
    308 ms    /controllers.UserController.annotations
    280 ms    /controllers.Authentication.handleRegistration
    226 ms    /controllers.TaskController.request

ReportController and StatisticsController is ok to be slow (for now).
Thumbnail is slow because hundreds of those are called at the exact same time, when the frontend decides its cache is gone (maybe paginate the gallery to send less thumbnail requests?). Upload/Download depends on individual size of tracing, also ok for now, I’d say.
Can we optimize the list queries? (Maybe with some caching / compactWrites omitting unused fields?)

[philippotto]

Btw, HTTP caching of the thumbnails seems to be missing completely. At least, chrome devtools never show "loaded from cache" for the thumbnails. We probably should set appropriate headers to nudge the browser to cache these.

• Double check HTTP caching for thumbnails

[philippotto]

Status from how I see it:

• dataset route was optimized
• thumbnail performance was improved by using better cache control

I'll move this back to backlog, @fm3. Feel free to close or re-manage.

[fm3]

Fine by me. Of course, other things can always be optimized but I don’t think there’s any acute problem :)

[fm3]

closing this since several of the slow calls have been optimized in the meantime :-) statistics and report pages are still slow, but this is expected and no one complained about that.