-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathprompts.txt
72 lines (66 loc) · 2.92 KB
/
prompts.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
The followings are conversations with an AI to complete tasks that require active information gathering from multimodalities. Otherwise, the materials of objects are unknown, and it will be ambiguous for an AI to choose the right object.
AI has the following skills to help complete a task:
1. "robot.knock_on()": to knock on any object and hear the sound to determine the material it consists of. Most of the materials can be determined by this skill.
2. "robot.touch()": to touch with haptics sensors. It is useful for some of the materials.
3. "robot.weigh()": to weigh objects if the knocking method is not proper.
4. "robot.pick_up()": to pick up the targeted object. After this skill is performed, the episode will terminate with result.
Note that the tasks are always set to be accomplishable, and the selected skill should start with a ">" symbol.
Human: "pick up the plastic cube" in a new scene that contains [blue block, red block, yellow block].
AI: > robot.knock_on(red block)
Human: It is made of metal.
AI: > robot.knock_on(blue block)
Human: It sounds resonant and ringing.
AI: > robot.touch(yellow block)
Human: It feels hard.
AI: > robot.knock_on(yellow block)
Human: It sounds dull.
AI: > robot.pick_up(blue block)
done()
Human: "pick up the glass block" in a new scene contains [yellow block on the left, blue block on the left, yellow block on the right, blue block on the right]
AI: > robot.weigh(yellow block on the left)
Human: It weighs light.
AI: > robot.touch(yellow block the right)
Human: It feels soft.
AI: > robot.weigh(blue block on the left)
Human: It weighs a little bit heavy.
AI: > robot.knock_on(blue block on the left)
Human: It sounds tinkling.
AI: > robot.touch(blue block on the left)
Human: It feels hard.
robot.pick_up(blue block on the left)
done()
Human: "pick up the metal block" in a new scene contains [red block, green block on the left, yellow block, green block on the right]
AI: > robot.knock_on(red block)
Human: Silent.
AI: > robot.knock_on(green block on the left)
Human: It sounds dull.
AI: > robot.knock_on(green block on the right)
Human: It sounds ringing.
AI: > robot.touch(green block on the right)
Human: It feels cold.
AI: robot.pick_up(green block on the right)
done()
Human: "pick up the fibre object" in a new scene that contains "[yellow block, blue block]".
AI: > robot.touch(blue block)
Human: It feels hard.
AI: > robot.touch(yellow block)
Human: It feels soft.
AI: > robot.weigh(yellow block)
Human: It weighs light.
AI: > robot.knock_on(yellow block)
Human: No obvious sound.
AI: > robot.pick_up(yellow)
done()
Human: "pick up the ceramic cube" in a new scene that contains "[blue block, red block, yellow block]".
AI: > robot.knock_on(red block)
Human: It is muted.
AI: > robot.touch(yellow block)
Human: It feels soft.
AI: > robot.knock_on(blue block)
Human: rattling sound.
AI: > robot.weigh(blue block)
Human: 100g.
AI: > robot.touch(blue block)
Human: It feels tough.
AI: > robot.pick_up(blue block)
done()