Data processing: generic control systems or specific application – Specific application – apparatus or process – Robot control
Reexamination Certificate
2002-04-29
2004-07-06
Cuchlinski, Jr., William A. (Department: 3661)
Data processing: generic control systems or specific application
Specific application, apparatus or process
Robot control
C700S031000, C700S248000, C700S258000, C700S259000, C318S568100, C318S568110, C318S568120, C318S568200, C318S569000, C901S001000, C901S015000, C901S047000, C704S207000, C704S209000, C704S270000
Reexamination Certificate
active
06760645
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to the solution of human-robot interaction problems and, more especially, to the training of robots, notably autonomous robots such as the animal-like robots that have recently come into use.
2. Description of Related Art Including Information Disclosed under 37 CFR 1.97 and 1.98
In recent years there has been an increase in the number of autonomous animal-like robots that have been developed and put on the market, such as Sony Corporation's four-legged AIBO™ robot, which resembles a dog—see “Development of an autonomous quadruped robot for robot entertainment” by M. Fujita and H. Kitano, in Autonomous Robots, 5, 1998. See also “Robots for kids: Exploring new technologies for learning”, by A. Drum and J. Hendler, Morgan Kaufman Publishers, 2000, and “The art of creating subjective reality: an analysis of Japanese digital pets” by M. Kusahara, in the Proceedings of the Artificial Life VII Workshop, 2000, ed. C. Maley and E. Boudreau, pages 141-144.
These autonomous robots are designed not as slaves programmed to follow commands without question, but as artificial creatures fulfilling their own drives. Part of the interest found in owning or interacting with such an autonomous robot is the impression the user receives that a relationship is being developed with a quasi-pet. However, autonomous robots can be likened to “wild” animals. The satisfaction that the user finds in interacting with the autonomous robot is enhanced if the user can “tame” the robot, to the extent that the user can induce the robot to perform certain desired behaviours on command and/or to direct its attention at, and learn the name of, a desired object.
To the user, it appears that he is “training” the robot, by analogy with human-animal interactions. However, given that the robot is more accurately be described as a kind of dynamic programming in the field. In the present document, references to “training” should be understood in this sense.
However, it is difficult to train an autonomous robot to perform specific tasks on command, especially tasks involving an unusual pattern of behaviour or a sequence of actions, or to learn the name for specific objects. Several groups are involved in research in this field, see, for example, “Experiments on human-robot communication with robota, an interactive learning and communicating doll robot.” by A. Billard, K. Dautenhahn and G. Hayes, from “Socially situated intelligence workshop” (SAB 98), eds. B. Edmonds and K. Dautenhahn, 1998, pages 4-16; “Experimental results of emotionally grounded symbol acquisition by four-legged robot” by M. Fujita, G. Costa, T. Takagi, R. Hasegawa, J. Yokono and H. Shimura, in the Proceedings of Autonomous Agents 2001, 2001; “Learning to behave: Interacting agents” by F. Kaplan, from the CELE-TWENTE Workshop on Language Technology, October 2000, pages 57-63; and “Learning from sights and sounds: a computational model” PhD thesis by D. Roy, MIT Media Laboratory, 1999.
The present inventors, considering that the problems involved in teaching a complex behaviour (and associated command) to an autonomous robot, and/or in reaching shared attention with an autonomous robot such that the name of a desired object could be taught, are similar to the problems faced by animal trainers, determined that robots could be trained by application of techniques used for pet training.
Over the last fifty years, there have been some fruitful exchanges between ethologists and robotics engineers. For example, in some cases robotics engineers have defined control architectures for robots, based on observations about animal behaviour. Different surveys of behaviour-based robotics are given in “Behaviour-based robotics” by R. Arkin, MIT Press, Cambridge Mass., USA, 1998; in “Understanding intelligence” by R. Pfeiffer and C. Sheier, MIT Press, Cambridge, Mass., USA, 1999; and in “The ‘artificial life’ route to ‘artificial intelligence’. Building situated embodied agents,” by L. Steels and R. Brooks, Lawrence Erlbaum Ass., New Haven, USA, 1994. Robot-based research has also led to development of models that may be useful for understanding animal behaviour—see “What does robotics offer animal behaviour?” by Barbara Webb, Animal Behaviour, 60:545-558, 2000. However, so far, when tackling robotics problems robotics researchers have not made many investigations in the field of animal training.
The method most often used by dog owners attempting to train their pets, for example, to sit down on command, involves chanting the command (here “SIT”) several times, whilst simultaneously forcing the animal to demonstrate the desired behaviour (here by pushing the dog's rear down to the ground). This method fails to give good results for various reasons. Firstly, the animal is forced to choose between paying attention to the trainer's repeated word, or to the behaviour to be learnt. Secondly, as the command is repeated several times, the animal does not know which part of its behaviour to associate with the command. Finally, very often the command is said before the behaviour is exhibited; for instanced “SIT” is said while the animal is still in a standing position. Thus, the animal cannot associate the command with the desired sitting position.
For these reasons, animal trainers usually one of the techniques listed below (which involve teaching a desired behaviour) first, and then add the associated command. The main techniques are:
the modelling method,
the luring method,
the capturing method,
the imitation method, and
shaping methods.
The present inventors considered that it was advisable to follow the same sort of approach when training a robot, given that the problem of sharing attention and discrimination stimuli is even more difficult with a robot than with an animal.
The modelling method is another technique often tried by dog owners but rarely adopted by professional trainers. This involves physically manipulating the animal into the desired position and then giving positive feedback when the position is achieved. Learning performance is poor, because the animal remains passive throughout the process. Modelling has been used in an industrial context to teach positions to non-autonomous robots. However, for autonomous robots which are constantly active, modelling is problematic. Only partial modelling could be envisaged. For instance, the robot would be able to sense that the trainer is pushing on its back and then decide to sit, if programmed to do so. However, it is hard to generalise this method to the training of complex movements involving more than just reaching a static position.
The luring method is similar to modelling except that it does not involve a physical contact with the animal. A toy or treat is put in front of the dog's nose and the trainer can use this to guide the animal into the desired position. This method gives satisfactory results with real dogs but can only be used for teaching position or very simple movement. Luring has not been used much in robotics. The AIBO™ robots that have been released commercially are programmed to be interested automatically in red objects. Some owners of these robots use this tendency so as to guide their artificial pet into desired places. However, this usage remains fairly limited.
In contrast to the modelling and luring methods, the capturing methods exploit behaviours that the animal produces spontaneously. For instance, every time a dog owner acknowledges his pet is in the desired position or performing the right behaviour this gives a positive reinforcement.
The present inventors investigated the suitability of a capturing technique for training autonomous robots, using a simple prototype. The robot was programmed to perform autonomously successive random behaviours, some of which corresponded to desired behaviours with which it was wished to associate a respective signal (for example, a word). Each time the robot spontaneously performed one of the desired behaviours the corresponding signal was pre
Kaplan Frederic
Oudeyer Pierre-Yves
Cuchlinski Jr. William A.
Frommer William S.
Frommer & Lawrence & Haug LLP
Marc McDieunel
Sony France S.A.
LandOfFree
Training of autonomous robots does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Training of autonomous robots, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Training of autonomous robots will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3228078