Method of reducing dimensionality of a set of attributes...

Details Method of reducing dimensionality of a set of attributes... Method of reducing dimensionality of a set of attributes...

2001-06-07

2004-05-11

Amsbury, Wayne (Department: 2171)

Data processing: database and file management or data structures

Database design

Data structure types

C707S793000, C707S793000, C707S793000

Reexamination Certificate

active

06735589

ABSTRACT:

FIELD OF THE INVENTION
This invention relates generally to a method of reducing dimensionality of a set of attributes used to characterize a sparse data set and, more particularly, a method of reducing dimensionality of a set of attributes based on the calculated variance of the data values associated with each of the attributes.
BACKGROUND ART
Data mining identifies and extracts relevant data in a computer accessed database. In certain applications, a data set in a database may be in tabular or matrix format wherein the rows of the matrix represent individual observations and the columns of the matrix represent various attributes of the data. The cells of the matrix contain data values. Each observation or row of the matrix contains a data value for each of the attributes, that is, if a matrix data set has m observations (rows) and n attributes (columns), there will be m×n data values. In many applications, the number of non-trivial (that is, non zero) values per observation is much smaller than n. An example of this phenomenon occurs when attributes represent products sold in a supermarket while observations represent individual customer's market baskets. Most customers purchase a very small fraction of all available products during a shopping trip, so the vast majority of entries in each row is zero. This condition is commonly referred to as the matrix being “sparse.” When this condition of a sparse matrix does not hold, the matrix is commonly referred to as being “dense.”
It is often advantageous to obtain a reduced set of k attributes to characterize data in the matrix where k<n. This will be referred to as reducing the dimensionality of the matrix data set. One technique that has been used to reduce the number of attributes characterizing a matrix data set is referred to as singular value decomposition (SVD). The SVD method generates a set of k equations (describing the new attributes) with each new attribute being a linear combination of the original n attributes. Disadvantages of the SVD method include:
1) computational complexity—the SVD method requires computation time, CT, on the order of CT=A*Q*k*log(n), where A is a constant, Q is the number of nonzero entries in a data matrix, k is number of attributes in the reduced set of attributes and n is the number of attributes in the original set of attributes;
2) results in a dense matrix—because each new attribute is a linear combination of the original attributes, the SVD method results in a matrix of data values that is dense; and
3) resulting data is nonintuitive—the results of applying the SVD method do not have an intuitive interpretation since each of the resulting k attributes is a linear combination of the original attributes. Thus, while originally each attribute corresponded to, for example, a particular product, a new attribute might be something like
“2.0*white bread−0.3*cheddar cheese+0.7*peanuts”. It is generally very difficult to extract the “meaning” of such an attribute.
Values of an attribute may be continuous (e.g., age, height, or weight of a respondent) or discrete (e.g., sex, race, year). Discrete attributes, that is, attributes having data values that are discrete variables, have a finite number of data values. Certain discrete attributes have only two data values (0 and 1) associated with the attribute, i.e., sex—male or female, where male=0 and female=1. Such discrete attributes will be referred to as dichotomous discrete attributes.
While the SVD method has several disadvantages, its major advantage is that it is a very effective methodology with regard to maintaining the distance structure between observations. Essentially, if the attribute data values associated with each observation are viewed as an n dimensional vector, the distance between two observations may be calculated as follows:
Define:
Observation no. 1: let the first data row, R1=[d11, d12, d13, d14, d15, . . . , d1n] where d11 is the data value for observation no. 1 and attribute no. 1, d12 is the data value for observation no. 1 and attribute no. 2, . . . , and d1n is the data value for observation no. 1 and attribute no. n.
Observation no. 2: let the second data row, R2=[d21, d22, d23, d24, d25, . . . , d2n] where d21 is the data value for observation no. 2 and attribute no. 1, d22 is the data value for observation no. 2 and attribute no. 2, . . . , and d2n is the data value for observation no. 2 and attribute no. n.
Calculate distance value (DIST 1−2) between the pair of observations nos. 1 and 2 as follows:
DIST 1−2=[(d11−d21)
2
+(d12−d22)
2
+(d13−d23)
2
+(d14−d24)
2
+ . . . +(d1n−d2n)
2
]
½
Assuming similar distance values are determined for each of the m*(m−1)/2 pairs of the m observations, the SVD method has been found to be “robust” in maintaining the distance structure between all pairs of observations while reducing the dimension of the attributes characterizing the data set from n attributes to k attributes. The SVD method is “robust” in maintain the distance structure between the data points in the following sense. Let the distortion of a data point (a row) be equal to the square of the difference between its original distance from the origin and its distance from the origin after dimensionality reduction. Then, among all possible dimensionality reductions to k dimensions from the n original dimensions, the SVD method minimizes the sum of the distortion over all points.
SUMMARY OF THE INVENTION
The present invention provides a dimensionality reduction method useful in sparse data sets that is effective and efficient when implemented in software for use in connection with data mining of a database management system. The dimensionality reduction method for use in a sparse database substantially preserves the distance structure between observations. Sparse data matrices are predominant in most real world data mining applications.
The attribute reduction method of the present invention may be used for both continuous and discrete attribute data and is best suited for matrix data sets that are sparse, that is, data sets that have a high proportion of zero values. Many real world marketing-related databases have a large number of discrete attributes that have dichotomous or two state data values most of which have zero values. For example, assume that a company sells a large variety of products and the company has established a matrix in its database to track customer purchases. If the rows of a matrix represent customers of a company and the attributes of the matrix correspond to whether or not a customer has purchased each of the products sold by the company with cell having a value of 1 if customer i has purchased product j and having a value of 0 if customer i has not purchased product j. In a matrix such as this, it could easily be the case that 90% or more of the cell data values or cell entries have a value of zero.
It is an object of this invention to provide a method of efficiently reducing the number of attributes that characterize set of data values in a data matrix D where the data matrix is sparse, that is, has a high proportion of zero values. The data matrix D is defined by an m×n matrix wherein the m rows of the matrix represent individual observations, e.g., a customer, an event or occurrence, a location, etc., and the n columns represent attributes of the observations. The cells of the matrix D contain specific data values for the associated observation and attribute. A reduced dimension matrix, Dnew, including m observations and k attributes (k<n) is desired. It is assumed that the magnitude of the value of k is set by a user or is arbitrarily set based on the number of attributes in the full scale matrix D.
One aspect of the present invention is a dimensionality reduction method of selecting k attributes from a set of n attributes to characterize data values in a data matrix D having m observations. The value of the data

Affiliated with

Also associated with

Rating

Method of reducing dimensionality of a set of attributes... does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Method of reducing dimensionality of a set of attributes..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method of reducing dimensionality of a set of attributes... will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3244908