Creating script for launching MTurk compensation tasks

[JackUrb]

Overview

A long-requested feature, this PR introduces a way to be able to launch compensation tasks for mturk workers directly by their worker ID. This can be useful if someone reaches out with a problem about a task, and you want to be able to pay out to resolve the issue. The task is launched without a live backend, so after running the script the task is live and can be completed.

cc @EricMichaelSmith, as I know this request has been asked by the ParlAI team at one point.

Implementation

One key consideration is that we want to tie any compensation to a task, and as such this script requires that you attach the pay to an existing task_name. In doing so, we can then create a unit for that task name, and then mark it as a soft_reject without tying any agent to it. This will include it in bookkeeping for the total cost of an assignment, but leave it out from any data collation scripts.

Under the hood, the script creates a task run for the compensation HIT, and a tied assignment and unit. Then it creates a HIT, marks it in the MTurkDatastore, then pushes an email to the worker. It also creates and assigns a qualification that should be unique to the worker, and sets that as a requirement for viewing/completing the task.

Testing

>>> python launch_makeup_hits.py
Please enter a task name for bookkeeping. This task name will be tied to the additional spend granted through this script, and should be the same as the task you originally launched that you now need to compensate for:
>> parlai_chat
Please enter an MTurkRequester name to use to bonus from:
>> MY_PRIVATE_REQUESTER_sandbox
You can now enter a worker id, amount, and reason for as many compensation tasks as you want to launch for this.
Enter a worker id to compensate. Leave blank to move on to launching: 
>> MY_PERSONAL_SANDBOX_USER
Enter the amount in dollars to pay out in this compensation task:
>> $0.50
Provide reason for launching this compensation task. This will be sent to the worker:
>> Test compensation task
Enter a worker id to compensate. Leave blank to move on to launching: 
>> 
You entered the following tasks:
[{'worker_id': 'MY_PERSONAL_SANDBOX_USER', 'amount': 0.5, 'reason': 'Test task #2'}]
Input anything to confirm and continue...
Creating qualification for MY_PERSONAL_SANDBOX_USER: compensation-for-MY_PERSONAL_SANDBOX_USER-on-parlai_chat....
Creating task run and data model components for this HIT
Created TaskRun(1368), Assignment(4101), and SandboxMTurkUnit(5832, None, created)...
Creating and deploying task on MTurk
Sending email to worker...

Email:

Task:

[EricMichaelSmith]

Yeah this is great to have - thanks for implementing it! The one thing I'd suggest is that it might be useful to be able to pass in a JSON with all of these inputs in order to do this programmatically, in order to avoid having to go through the CLI in order to compensate 10 Turkers, say

[JackUrb]

Agreed that a json input would be nice, though I'll leave that as a TODO for someone who wants to pipe these inputs in the future.