Robot learns to track itself and the world through a mirror

Justin Hart / Yale University

Nico examines himself and his surroundings in the mirror.

A robot at Yale has passed an important milestone of self-awareness by inferring information about the real world by observing it through a mirror. It's not quite WALL-E, but it's an notable step in the world of artificial intelligence.
Nico is an ongoing project at Yale's Social Robotics Lab that has been advancing slowly for years; development is currently being led by roboticist Justin Hart under the supervision of his doctoral advisor Brian Scassellati. The latest accomplishment is subtle but interesting, and certainly tricky. It has to do with how a mirror is perceived.
Many of us have been delighted when a pet fails to recognize itself in the mirror and bristles or barks, thinking that the mirror is in fact an extension of a room and there is another animal approaching through it. We understand, however, that it is a reflection, and we could determine, for example, where a lamp is relative to ourselves by looking at the mirror but not the lamp itself.
What Nico (or rather, its creators) has accomplished is this latter task: it looks in the mirror and it sees its hand, which it recognizes because it has a visual token attached. From here, it learns basic expectations of where to see its hand in normal space. Then it uses that information to build estimates of where other objects in the room are — all without looking around at the world itself.
Hart explained in an email to NBC News that this ability is far from common in animals, since it requires a certain level of understanding about what mirrors actually are:

What makes this exciting, in terms of self-awareness, is that the robot is able to use this knowledge that it has learned about itself in order to reason about a thing in its environment, the mirror, in a way that robots really haven't been able to do before.
His ultimate goal, he says, is to have the robot pass an iconic test first proposed in 1970 involving an animal (or in this case, a robot) recognizing changes to its appearance in a mirror. By noticing that a change has been made to its appearance by looking into a mirror, the subject shows conclusively that it understands that the image is not another animal, but itself.

Dan Leyzberg / Yale University

Hart posing with Nico.

Research along these lines could lead to improved robotic vision and motion systems that are not just mechanically aware of their position, but also understand the world in a more intuitive way. A sense of "self" and "other" is fundamental to humans' everyday interactions, and if robots are to work closely with people, they should be able to make similar distinctions.
The paper, "Mirror Perspective-Taking with a Humanoid Robot," was presented last month at the Conference on Artificial Intelligence in Toronto, Canada, and Hart expects to put Nico through the second mirror test in the next few months.
Source:NBC News Digital