Bring Azure machinery auto-scaling up to date.

[ChrisThibodeaux]

Updated the Azure machinery, specifically with auto-scaling, to a usable state out-of-the-box.

Updates include:

• Config key to allow the selection of ephemeral OS disk type, allowing a wider range of Azure VM types to be used.
• Config key to allow changing the timer set for the monitor thread to scale up/down the VMSS.
• Fix for the missing machines key in the config, which was leading to errors.
• Change the method of finding "relevant" machines for submitted tasks, allowing more flexibility in naming your VMSS in the config. This removes the need to append the pool_tag to the end of the VMSS name.
• Workaround to semaphore deadlocking issues.
• Removal of machine deletion in the modules/machinery/az.py method stop when the scale-set is set to is_scaling_down, which was leading to a StaleDataError.
• Tiny, non-critical, update to a log in lib/cuckoo/core/guest.py correcting the order of variables.

Concerns:
The fix for the StaleDataError is not ideal. It means that the process of scaling down when no relevant tasks are available must wait for all current tasks to finish AND for the monitor thread to wake up and run. What would be a much better fix would be to find a way to signal the machine for deletion in az.py immediately after the AnalysisManager releases the machine. Any tips are greatly appreciated on how that might be done. (Edit: This no longer applies. Now correctly deleting machines in such cases.)

If there are any ideas on how to better handle the issue with the semaphore, I would love to better apply a solution there. I am not entirely comfortable with what I have in place here. The main issues leading to deadlocking that I have noticed are:

1. In update_limit -- The _limit_value is only updated when there are machines, and the amount of machines is less than the _upper_limit. So no update for _limit_value occurs when we are at the upper limit.
2. In check_for_starvation -- The available_count is the number of unlocked machines. If machines are waiting at the semaphore, they are already locked. This makes updating the _value here impossible.

As a check for releasing the semaphore, I used the condition of locked machines <= configured machine limit, but I think that should maybe be total machines (locked or unlocked) <= configured machine limit.

Thoughts?

[doomedraven]

let me know when is ready to merge

[ChrisThibodeaux]

@doomedraven I've tested trying to get the semaphore to lock up in various situations, but it looks good now. Should be good to merge.

[ChrisThibodeaux]

@doomedraven I believe I have found a better way of managing machine deletion. Please hold off on the merge until I can get the commits together.

[ChrisThibodeaux]

@doomedraven Okay, fix is in. Now operating as expected.

[doomedraven]

thank you for update