06. Tutorial 6

In this tutorial, we introduce the keyword 'targeting'.

This keyword is used to set sensory motor loop during runtime (e.g. to have the robot walking toward a moving ball while looking at it).

This is the most powerful keyword of playful. But it is counter intuitive (you have been warned).

Playful program

program:

	virtual_balls_detection
	targeting ball: ball_display

The node "virtual_balls_detection" pushes information about a blue and a green bouncing balls to playful's memory. For each ball, Playful spawns an instance of "ball_display", resulting in the display of all the balls (here the blue and the green).

Intuition

To give an intuition where this is going at: Imagine you want the robot to always walk to the closest ball it detects, but you do not know in advance how many balls will be detected by the robot. Because of 'targeting', this could be implemented in a single command:

     targeting ball: walk_to, priority of 1/distance

The code above sets the following:

• Everytime a new ball is detected at runtime, the playful engine spawns a new walk_to action.
• All the nodes and evaluations (here walk_to and distance) refer to this new ball instance (it 'targets' this specific ball), i.e. it sets a sensory motor loop between ball detection and update of walk configuration.
• At runtime, several instances of "walk_to" (one per detected ball) may exist, and compete for activation. The prioritization ensures the robot always walks toward the closest one (as continuously reevaluated).

Note that at startup, before the robot detects any ball, no "walk_to" action is instantiated.

In this tutorial we will see:

• how to define a scheme that can be targeted ('ball' here is called a 'scheme', using playful terminology).
• how to create nodes and evaluations that can be targeted by a new scheme instance.
• how to define what "new" means, as in "the robot was tracking a ball, turned its head and saw a new ball".

_{Optional explanations about "how to define what new means". For each new ball the robot detects, playful should spawn a new walk action. But imagine the robot sees a red ball, turns the head, and sees a red ball again. Did the robot saw only one ball, which moved, or did the robot saw two different balls ? Playful is agnostic to this. The developer is in charge to set the rule. For example, the developer of a soccer playing robot knows that, by convention, there is only one ball. Other environments may be more complex. In Playful, the developer encodes the rules of what "new" means by implementing similarity functions. In short: if two schemes are similar, then they are actually the same. If the robot sees a ball, then sees a very similar ball, it actually saw twice the same ball, and no new walking action is spawned. This may seem confusing for now, but for most cases, this is trivial to implement. This tutorial shows an example that will work for most of the time. The trick is that for most application, there is a property of the scheme that closes the debate. In this tutorial, it will be the color, i.e. if two balls are of the same color, they are the same ball.}

Python code for properties

The playful code mentions a ball:

targeting ball: ball_display

Using playful terminology, "ball" is a scheme. A scheme is a collection of properties, in this case a position, a time stamp and a color. Related python code:

class position(playful.Property):

    def fuse(self,value):
        self._value=value

    def similarity(self,value):
        return None

class time_stamp(playful.Property):

    def fuse(self,value):
        self._value=value

    def similarity(self,value):
        return None

class color(playful.Property):

    def fuse(self,value):
        self._value=value

    def similarity(self,value):
        if self._value == value :
            return True
        return False

Playful properties, the building blocks of schemes, are just a value holder (self._value), and fuse and similarity functions. In short, these properties make the assumption here that there will be a unique ball for each color (only the similarity function of color returns True if the color value is identical).

_{Optional explanations: Fuse functions set how the property of a scheme should be updated based on new perception. Overriding the value as in this tutorial is typically ok. The similarity functions is used to evaluate the similarity between two schemes. The "None" value returned by the functions of position and time_stamp indicate these properties have no influence on the measurement of similarity. On the other hand, the similarity function of color indicates that two schemes of the same color correspond to the same scheme.}

Playful code for ball

The ball scheme is declared as a position, time_stamp and color in the file /playful_tutorial/play/schemes.play.

ball : scheme
      position, time_stamp, color

It is trivial to create new properties and declare new types of scheme.

Playful node for ball detection

Our simulated robot does not have any real sensors, and for this tutorial we simply create a node which simulates the detections of bouncing balls

class virtual_balls_detection(playful.Node):

    def execute(self):

        # simulating bouncing balls in 1D
        class _Position_manager:

            def __init__(self,initial_position,speed,direction=1,max_x=30):
                self.position = initial_position
                self.speed = speed
                self.direction = direction
                self.max_x = max_x

            def update(self):
                self.position += self.direction * self.speed
                if self.position > self.max_x :
                    self.position = self.max_x
                    self.direction = -1 * self.direction
                if self.position < 0 :
                    self.position = 0
                    self.direction = -1 * self.direction
                return self.position
                
        time_start = time.time()

        blue = _Position_manager(0,0.2)
        green = _Position_manager(0,0.1)
        #you may uncomment for fun
        #red = _Position_manager(10,0.15)
        
        while not self.should_pause():

            t = time.time()

            # creating a ball objects from properties, and sending it to the memory
            # "fuse" means the memory compares the incoming scheme with schemes already
            # maintained in the memory (similarity functions of the properties) and either maintain a new
            # scheme or fuse with an existing scheme (fuse functions of the properties)

            # the blue ball is created from start
            playful.memory.fuse(playful.create("ball",position=blue.update(),time_stamp=t,color="BLUE"))

            # the green ball is created 4 seconds after start
            if(t-time_start > 4):
                playful.memory.fuse(playful.create("ball",position=green.update(),time_stamp=t,color="GREEN"))

            # uncomment for fun
            #playful.memory.fuse(playful.create("ball",position=red.update(),time_stamp=t,color="RED"))

            self.spin(20)

This leaf node simulates the perception by robot of virtual balls (in real applications, this could be a node using for example opencv over a video stream to detect balls, determining their color, computing their position, and pushing related ball schemes to the memory)

This node simulates:

• from the start, a blue ball is detected
• a green ball is detected 4s after this.

This node encodes info into a ball schemes that it pushes to the memory. The memory uses the similarity and fusing functions to update existing schemes and create new ones.

When running the tutorial, you may see the program expanding during runtime after 4 seconds : the targeting keyword (in the playful script) commands a new leaf node corresponding to the green ball to be created.

Also, feel free to uncomment code in the node to simulate a 3rd red ball, and notice the playful script does not need to be updated. The playful code is agnostic to the number of balls detected.

Playful node for displaying a ball

The python code simply read from the memory the value of the properties of the targeted scheme, and display it using Playful's console.

# class for displaying a ball in the console
# (note that the code is not specific to ball, though:
# it is specific to scheme with a color and position properties)

class ball_display(playful.Node):

    def execute(self):

        while not self.should_pause():
        
            # the targeting keyword in the playful script associated this instance of
            # 'ball_display' with one of the ball scheme. 'self.get_scheme_id()'
            # returns the unique id of the ball scheme this instance is associated to,
            # and the get_propery_value function allows to access the properties of this scheme.
            position = playful.memory.get_property_value("position",scheme_id=self.get_scheme_id())
            color = playful.memory.get_property_value("color",scheme_id=self.get_scheme_id())

            if position and color :
                playful.console(str(id(self))," "*int(position)+str(color))
            
            self.spin(10)