Data processing: artificial intelligence – Neural network – Learning task
Reexamination Certificate
1998-08-07
2001-06-12
Powell, Mark R. (Department: 2122)
Data processing: artificial intelligence
Neural network
Learning task
C706S048000, C706S052000
Reexamination Certificate
active
06247002
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to the categorisation or characterisation performed by an agent (typically a robot or computer system) on the basis of perceived properties or qualities and, notably, the invention relates to the extraction of features characterising objects. The invention also concerns a system for autonomous generation of a communications protocol, using this feature extraction technique.
The present invention consists in a mechanism for the automatic and spontaneous formation of “meanings”, “descriptions” or “categories”, which are perceptually grounded, under selectionist pressure arising from a discrimination task. The established “descriptions” can be used for a number of purposes, including communication.
2. Description of the Related Art
A discrimination task demands the ability to distinguish one entity (such as an object), which constitutes the topic, from a series of other entities, which constitute the context. Discrimination takes the form of determining a description of the topic which distinguishes it from the context. The description of the topic will involve the identification of a combination of “categories” or “features” which apply to the topic but not to the entities making up the context. The present invention provides a mechanism for constructing a distinctive feature set from data points representative of an entity, which mechanism autonomously develops the required “categories” or “features”.
The invention has arisen in the context of a larger research program to understand the origins of language and meaning using complex systems mechanisms such as self-organisation, co-evolution and level formation (see “Synthesising the origins of language and meaning using co-evolution and self-organisation” by L. Steels in “Evolution of Human Language” edited by J. Hurford, Edinburgh University Press, Edinburgh, 1996c).
The invention concerns the “meaning creation” process, that is, the technique whereby the agent derives characterisation rules and/or information. A method and apparatus are proposed whereby an autonomous agent may originate new meanings. The agent is autonomous in the sense that its ontology is not explicitly put in by a designer, nor is there any explicit instruction.
For the purposes of the present document, meaning is defined as a conceptualisation or categorisation of reality which is relevant from the viewpoint of the agent. Meanings or categories can be expressed through language, although they need not be.
In very general terms, meaning takes many forms depending on the context and nature of the situation concerned. Some meanings (such as colours) are perceptually grounded. Others (such as social hierarchies) are grounded in social relations. Still others (such as goals or intentions for actions) are grounded in the behavioural interaction between the agent and the environment. The present invention focuses on perceptually grounded meanings, although the proposed mechanism could also be used for other domains.
The proposed method and apparatus may be employed in a wide variety of different applications where agents (e.g. software agents or robotic agents) autonomously have to make sense of their environment. This ability is especially important in the case of devices, such as unmanned exploration vehicles, which are required to navigate and perform tasks in, and/or gather data concerning, a little-known and/or hostile environment.
In a more general way, the present invention may be applied with advantage in numerous other fields where it is necessary to be able to characterise entities, for example, natural language processing systems, character or voice recognition systems, expert problem solving systems, systems indexing objects based on images (whether for creation of an index for a database of the images or for operational purposes, such as, the detection of cracks in an imaged structure), etc.
The present invention concerns both “meaning creation” in a single agent, and “meaning creation” in multiple agents which communicate with one another using the categories that they have developed. In the latter case, a common communications protocol or “language” can be built up and can act as a way to achieve a coherent conceptual framework between agents even though every agent individually builds up its own repertoire.
There is no suggestion that the method according to the present invention corresponds empirically to any processes performed in the animal or human mind. This method does, however, enable artificial devices to create meaningful characterisation information in a wide variety of applications.
There has been a lot of work on the problem of meaning creation particularly in the connectionist literature (see, for example, “Explorations in Parallel Distributed Processing” edited by J. L. McClelland and D. E. Rumelhart, MIT Press/Bradford Books, Cambridge, Massachusetts, 1986). A perceptron, for example, can be seen as a device that acquires a set of distinctions as relevant for a classification task. The sensory channels are constituted by the inputs to the perceptron, and the weights perform the function of selecting out regions which will be input for the classification process.
The most important difference between these connectionist approaches and the present invention is that:
(1) connectionist networks embed the build-up of a feature repertoire within the task of classification (as opposed to discrimination), and
(2) an inductive/instructional approach as opposed to a selectionist approach is used.
An inductive approach is based on going through a (typically large) set of examples which drives the weights stepwise to reflect the best classification. In a selectionist approach, a structure comes into existence by variation or construction and is then tested as a whole for fitness in the environment. Inductive approaches result in gradual generalisation. Selectionism immediately gives generalisations which might be refined more gradually.
The selectionist approach followed in the present invention is more in tune with work on feature generation in genetic algorithms research (see, for example, “Genetic Programming” by J. Koza, MIT Press, Cambridge, Mass., 1992), unsupervised learning as exemplified by the Kohonen network (see “Self-Organization and Associative Memory” by T. Kohonen, Springer Series in Information Sciences, vol.8, Springer Verlag, Berlin), and proposals, known as “Neural Darwinism”, made by Edelman (see “Neural Darwinism: The Theory of Neural Group Selection” by G. M. Edelman, Basic Books, New York, 1987).
Edelman assumes that neuronal growth processes yield a primary repertoire stabilised by developmental selection, which is then subjected to experiential selection, yielding a secondary repertoire of categories. Using re-entrant maps and degeneracy, categorial perceptions of different objects can be compared and generalised to classes. Meaning creation and classification are clearly distinct here. The selectionist pressure in the case considered by Edelman comes from statistical signal correlations (for the formation of classes). By way of contrast, in the present invention, the selectionist pressure comes from a discrimination task.
In summary, the primary disadvantages of the prior art techniques consist in the fact that the categories are supplied by the designer, the categories are not based upon perception, a series of examples is needed in order to “train” the system, there are distinct “training” and “use” phases such that new categories cannot be developed outside the “training” phase, and the data channels are fixed.
SUMMARY OF THE INVENTION
The present invention provides a method and apparatus for extracting features of entities, particularly objects, avoiding the above-mentioned disadvantages of the prior art. The present invention provides, in addition, a technique for autonomous development of a communications protocol or language, using the feature extraction method.
Considering, first, the invention in general terms, it should be n
Frommer William S.
Frommer & Lawrence & Haug LLP
Khatri Anil
Powell Mark R.
Savit Glenn F.
LandOfFree
Method and apparatus for extracting features characterizing... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for extracting features characterizing..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for extracting features characterizing... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2475114