Scheduler /queue and worker

[butonic]

Discussing chunked upload with @labkode we iterated over the concept of a scheduler to handle asynchronous tasks.

When the last cunk of a file has ben added an assemble task is added to a queue. A worker willl take the task, assemble the file and produce the final metadata in the storage.

We can use the queue to implement metadata (etag and mtime) propagation fkr storages that do not support it natively (all but eos and another owncloud mounted via webdav).

But we need to clarify how to deal with worker crashes. One possibility is to not only push an event/task to the queue but also create an event/task file on disk. The worker can modify the event and even move them to an activity log. The activity log is tightly related. We want to be able to show the list of activities for a user. We could use these events on disk for persistency. Any crashed worker can just be restarted and his task can be readded.

in any case we want to keep the queue implementation simple. It will crash. It is more important to be able to aggregate events and be resilient.