Use tokio::sync::Semaphore to wait for available task slots

[tfeda]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

@Dandandan, I think we could improve on #378 by using tokio::sync::Semaphore to wait for task slots to open up, rather than sleep spinning. It could look something like this:

let available_task_slots = Arc::new(Semaphore::new(num_task_slots))

loop {
   // thread sleeps until permits are available from spawned tasks
   let task_permit = available_task_slots.acquire_owned().await  
    
   // poll scheduler for work

  if work_is_available {
     tokio::spawn(async move {
       // task work  
   
       // move permit into task scope
       drop(task_permit)
     })
  }
  // If no work is available, task_permit gets dropped here
}

Looking at the current implementation, its pretty much the same thing as what a tokio semaphore does, except with the sleeping spin loop.

Describe alternatives you've considered

Just a thought, implementing this isn't a deal breaker

[Dandandan]

@tfeda thanks... that seems what we are looking for.
Could save some extra milliseconds between tasks 👍