Make worker status responses threadsafe

[daveyarwood]

I created a brittle "worker status" system based on mutable state and relying on the client to make requests in a certain way.

Now that we're working on a new client-side REPL (alda-lang/alda-client-java#1), bugs with this approach are starting to surface. It's becoming obvious that I need to re-think how this system works and make it more reliable.

How it works now

• Worker is in the :available state, waiting for work.

• Client makes a request to the server to play a score, and the server hands the request to the worker.

• The worker goes into the :parsing state while it parses the score. If it gets a status request from the client during this time, it says it's parsing the score.

• If the score is parsed successfully, the worker has a score in memory. It goes into the :playing state and starts to play the score. If it gets a status request during this time, it says it's playing the score and includes the score in the response.

  • When it's done playing the score, the worker goes into the :available state and continues to have the score in memory and return it in status responses, in case the client isn't quick enough to send the status request and it still needs the score.

• If the score is not parsed successfully, the worker goes into the :error state and holds the error in memory. If it gets a status request while in this state, it says there was an error and sends the error back so the client can handle it.

Problem

I think the problem with this is that when playing tiny scores in rapid succession (e.g. evaluating lines of code in the Alda REPL), status requests can come from the client in an arbitrary order, and potentially overlap with status requests for other lines of code (scores) submitted to be played. The worker can only really be in the state of handling one score at a time, so it's possible to send the wrong data sometimes.

Solution

Associate each parse/play job with a unique ID, and store a number (say, 10) of previous results (state, parsed score and/or error) in a map keyed by the job ID. Make these results available to the client at any time regardless of whether a new job has started; the client just needs to provide the job ID to get the status of that score being parsed/played.

To make this backwards compatible, we can default to the most recent job, if no job ID is provided. This is the current behavior.

An added benefit is that this sets us up for clients to be able to pause and stop the playback of individual jobs ( alda-lang/alda-sound-engine-clj#3 ).

I think a FIFO cache from clojure/core.cache might work for the map of job ID => status. We can set the threshold at 10 (or whatever) and the cache will only hold up to that many results and discard the old ones.

[daveyarwood]

Fixed in 0.2.0.