Use semaphore for queue lock

[jkemp101]

• Add tests for max workers > 1
• Test Redis impact with large number of locked queues, large number of workers, and new tasks being queued to locked queues

[tsx]

We could add an optional def acquire(self, time=time.time): parameter, pass time explicitly in tests and avoid freezefrog dependency.

[thomasst]

Should there be an option to configure this per queue, and then deprecate SINGLE_WORKER_QUEUES? E.g.:

MAX_WORKERS_PER_QUEUE = {
    None: 4,    # default
    'some-queue': 1,
    'some-queue.subqueue': 2,
    'other-queue': 6,
}

[jkemp101]

I'm a bit torn on how we should handle queue configuration as we add more and more queue specific options. Right now a queue's configuration is spread across multiple keys in the config. Do you think we should consider moving to something like this instead and stop using ONLY_QUEUES, BATCH_QUEUES, SINGLE_WORKER_QUEUES/MAX_WORKERS_PER_QUEUE?

QUEUES = {
    'a': {
        max_workers: 5,
        batch: 50
    },
    'b': {}
}

We could also make it smart enough to detect if ONLY_QUEUES is a list or a dictionary so we don't have to add QUEUES to the config.

If we wanted to make that change I could remove MAX_WORKERS_PER_QUEUE for now and just support it as a command line option until we refactor the config handling.

[thomasst]

Is this used anywhere?