Data processing: database and file management or data structures – Database design – Data structure types
Reexamination Certificate
1999-06-10
2001-09-11
Amsbury, Wayne (Department: 2171)
Data processing: database and file management or data structures
Database design
Data structure types
C707S793000, C707S793000
Reexamination Certificate
active
06289353
ABSTRACT:
BACKGROUND OF THE INVENTION
This invention relates to accessing information and categorizing users and more particularly to an adaptive and scalable indexing scheme.
Document retrieval often involves accessing a large information space. This information space is characterized by many dimensions. Each document occupies a single point in this information space. However, the organization of documents in the space is complex. This complexity is a product of the dimensionality of the space. Documents share properties, and thus share the coordinates of some subset of dimensions, but differ with respect to other properties. Because of this, the entire information space is only sparsely populated with documents. Sparse distribution of documents in the information space makes intelligent searching of the space difficult. The relationships between two documents are only poorly described in the space since the documents typically differ in more ways than they are the same. Across a group of documents, there is minimal structure to organize a search for relevant documents.
Artificial neural networks (ANNs) are used to generate statistical relationships among the input and output elements, and do so thorough self-organization or, at least, through an automated abstraction or learning process. Several efforts have employed ANNs to a limited extent for information retrieval. The ANN contains a set of constraints which, when given some input pattern coding a query, directs the user to similar documents or pieces of information. The initial set of constraints is generally determined by the application of a training corpus set of records to the ANN. These constraints are incrementally modifiable, allowing the ANN to adapt to user feedback. However, although several research efforts have demonstrated the utility of adaptive information retrieval with ANNs, scalable implementations have not appeared. For reviews, see Doszkocs, 1990, and Chen, 1995, incorporated herein by reference.
On the other hand, some large-scale systems which lack mechanisms for adaptation have successfully exploited the statistical relationships among, documents and terms found in those documents, for storage and retrieval of documents and other information items. For example, U.S. Pat. No. 5,619,709 to Caid, et. al., describes generation of context vectors that represent conceptual relationships among information items. The context vectors in Caid, et. al. are developed based on word proximity in a static training corpus. The context vectors do not adapt to user profile information, new information sources, or user feedback regarding the relevancy of documents retrieved by the system. Thus, the system in Caid, et. al. does not evolve over time to provide more relevant document retrieval.
Accordingly, a need remains for a scalable information representation and indexing scheme that adapts document retrieval to continuously changing user feedback, user profiles, and new sources of information.
SUMMARY OF THE INVENTION
An Intelligent Query Engine (IQE) system automatically develops multiple information spaces in which different types of real-world objects (e.g., documents, users, products) can be represented. The system then delivers information to users based upon similarity measures applied to the representations of the objects in these information spaces. The system simultaneously classifies documents, users, products, and other objects. Any object which can be related to or represented by a document (a chunk of text) can participate in the information spaces and can become the target of similarity metrics applied to the spaces.
The system automatically indexes large quantities of documents in a database. The indices are managed by persistent objects known as collators. Collators are resident in the system and act as classifiers of overlapping portions of the database of documents. Collators evolve to meet the demands for information delivery expressed by user feedback. Collators evolve under selective pressure to cover as much of the database as possible under the constraints of finite and particular computing resources. Other objects, known as liaisons, act on the behalf of users to elicit information from the population of collators. This information is then presented to users upon logging into the system via Internet or another communication channel. Object-oriented programming facilitates the implementation of a highly distributed system of asynchronously communicating liaisons and collators.
Collators propagate in the system via success at attracting and delivering relevant information to users. Thus, not only are there multiple information spaces, but these are competing ways of representing the universe of information elements. An evolutionary model is applied to the system to optimize the allocation of resources to collators and to promote specialization among the population of collators. That is, the evolutionary framework makes the system scalable by establishing the criteria that determine which documents are good documents and which documents can be ignored or removed. The evolutionary framework also makes the system more effective at locating the most relevant documents by refining the semantic structure generated through retention of good documents.
Objects called mites handle incoming documents from multiple information sources (e.g., in-house editorial staff, third-party news feeds, large databases, World Wide Web spiders) and feed documents to those collators which provide a good fit for the new documents. Mites recycle documents from collators that are removed from the system due to inability to satisfy the information needs of users. Mites also archive documents from the database which fail to fit well with any collators.
Liaisons act on behalf of the users to retrieve information via the views of the database provided by collators. These views provide interpretations of all of the participating objects: documents, users represented by the documents they have read and rated as relevant, products represented by documents, etc. The system thus provides a mechanism for delivering relevant documents, putting users in touch with other users who have similar reading interests, and recommending relevant products to users.
Machine learning techniques are used to facilitate automated emergence of useful mathematical spaces in which information elements are represented as vectors of real numbers. A first machine learning technique automatically generates a set of axes that characterize the central semantic dimensions of a collator's set of documents. The procedure begins with the set of documents coded as vectors of term frequencies in an information space spanned by a dictionary of all terms in the set. The collator then finds a reduced dimensionality space spanned by a set of concepts which are central to a significant portion of the set of documents. The original information space, spanned by the entire dictionary, is mapped into a low-dimensional space spanned by a set of central concepts. The new low-dimensional space represents a particular view of the portion of the database represented by the collator's set of documents. The database portion is not chosen in advance, but evolves contemporaneously with the vector space structure which emerges.
The collators operate as classifiers in an evolutionary framework. The particular vector spaces developed by collators, as described above, are subject to two kinds of selective pressure. First, the vector space must provide a good fit to many documents. Second, the vector space must provide delivery of relevant documents to many users. The first kind of fitness is measured directly from the ability of the reduced dimensionality vector space to code documents made available by mites. The second kind of fitness is derived from user feedback. Explicit and implicit user feedback is used to identify successful collators. Fit collators propagate their vector spaces into the next generation via reproduction while unfit collators are eliminated.
The system utiliz
Burke Scott M.
Hazlehurst Brian L.
Nybakken Kristopher E.
Amsbury Wayne
Conley Rose & Tayon PC
Pardo Thuy
Webmd Corporation
LandOfFree
Intelligent query system for automatically indexing in a... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Intelligent query system for automatically indexing in a..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Intelligent query system for automatically indexing in a... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2533065