Skip to content

Commit

Permalink
Limit request_task concurrency
Browse files Browse the repository at this point in the history
A request_task call takes about 150ms (7 per second).

The worker randomizes calls in a 1 to 30s interval.

With 200 workers or more the request queue might consume
all threads, so limit concurrency to 4 requests.
  • Loading branch information
tomtor authored and Stefano80 committed Apr 6, 2018
1 parent b2528b5 commit 2c9e381
Showing 1 changed file with 8 additions and 2 deletions.
10 changes: 8 additions & 2 deletions fishtest/fishtest/rundb.py
Original file line number Diff line number Diff line change
Expand Up @@ -266,14 +266,20 @@ def get_results(self, run, save_run = True):

# Limit concurrent request_task
task_lock = threading.Lock()
task_semaphore = threading.Semaphore(4)

task_time = 0
task_runs = None
worker_task = {}

def request_task(self, worker_info):
with self.task_lock:
return self.sync_request_task(worker_info)
if self.task_semaphore.acquire(False):
with self.task_lock:
r= self.sync_request_task(worker_info)
self.task_semaphore.release()
return r
else:
return {'task_waiting': False}

def sync_request_task(self, worker_info):

Expand Down

0 comments on commit 2c9e381

Please sign in to comment.