fix race condition in generate_units

[PReithofer]

Enumerating over unlaunched_units.items() can cause a runtime error if the size of unlaunched_units changes during iteration. This can be avoided by acquiring the lock earlier.

[PReithofer]

The concrete error I get is:

Exception in thread unit-generator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/workspaces/Mephisto/mephisto/operations/task_launcher.py", line 184, in _launch_limited_units
    for unit in self.generate_units():
  File "/workspaces/Mephisto/mephisto/operations/task_launcher.py", line 161, in generate_units
    for i, item in enumerate(self.unlaunched_units.items()):
RuntimeError: dictionary changed size during iteration

I made some minimal changes to the static_react_example to reproduce this bug. You can check it out here:

https://github.com/PReithofer/Mephisto/tree/dict-size-change-bug

[JackUrb]

Thanks for this fix @PReithofer, makes complete sense! Launching with generators remains a functionality that is admittedly under-tested, good to see issues discovered and resolved.