Distributed content identification system

Details Distributed content identification system Distributed content identification system

1999-12-22

2002-10-01

Metjahic, Safet (Department: 2171)

Data processing: database and file management or data structures

Database design

Data structure types

C707S793000, C709S203000, C709S206000

Reexamination Certificate

active

06460050

ABSTRACT:

BACKGROUND OF THE INVENTION
1. Field of the Invention
The invention relates to the field of content identification for files on a network.
2. Description of the Related Art
With the proliferation and growth of the Internet, content transfer between systems on both public and private networks has increased exponentially. While the Internet has brought a good deal of information to a large number of people in a relatively inexpensive manner, this proliferation has certain downsides. One such downside, associated with the growth of e-mail in particular, is generally referred to as “spam” e-mail. Spam e-mail is unsolicited e-mail which is usually sent out in large volumes over a short period of time with the intent of inducing the recipient into availing themselves of sales opportunities or “get rich quick” schemes.
To rid themselves of spam, users may resort to a number of techniques. The most common is simple filtering using e-mail filtering which is built into e-mail client programs. In this type of filtering, the user will set up filters based on specific words, subject lines, source addresses, senders or other variables, and the e-mail client will process the incoming e-mail when it is received, or at the server level, and take some action depending upon the manner in which the filter is defined.
More elaborate e-mail filtering services have been established where, for a nominal fee, off-site filtering will be performed at a remote site. In one system, e-mails are forwarded offsite to a service provider and the automatic filtering occurs at the provider's location based on heuristics which are updated by the service provider. In other systems, offsite filtering occurs using actual people to read through e-mails and judge whether e-mail is spam or not. Other systems are hybrids, where heuristics are used and, periodically, real people review e-mails which are forwarded to the service to determine whether the e-mail constitutes “spam” within the aforementioned definition. In these hybrid services, personal reviews occur on a random basis and hence constitute only a spot check of the entire volume of e-mail which is received by the service. In systems where real people review e-mails, confidentiality issues arise since e-mails are reviewed by a third party who may or may not be under an obligation of confidentiality to the sender or recipient of the e-mail.
In addition, forwarding the entire e-mail including attachments to an outside service represents a high bandwidth issue since effectively this increases the bandwidth for a particular e-mail by three times: once for the initial transmission, the second time for the transmission to the service and the third time from the service back to server for redistribution to the ultimate recipient.
Further, senders of spam have become much more sophisticated at avoiding the aforementioned filters. The use of dynamic addressing schemes, very long-length subject lines and anonymous re-routing services makes it increasingly difficult for normal filtering schemes, and even the heuristics-based services discussed above, to remain constantly up-to-date with respect to the spammers' ever changing methods.
Another downside to the proliferation of the Internet is that it is a very efficient mechanism for delivering computer viruses to a great number of people. Virus identification is generally limited to programs which run and reside on the individual computer or server in a particular enterprise and which regularly scan files and e-mail attachments for known viruses using a number of techniques.
SUMMARY OF THE INVENTION
Hence, the object of the invention is to provide a content classification system which identifies content in an efficient, up-to-date manner.
The further object of the invention is to leverage the content received by other users of the classification system to determine the characteristic of the content.
Another object of the invention is to provide a service which quickly and efficiently identifies a characteristic of the content of a given transmission on a network at the request of the recipient.
Another object of the invention is to provide the above objects in a confidential manner.
A still further object is to provide a system which operates with low bandwidth.
These and other objects of the invention are provided in the present invention. The invention, roughly described, comprises a file content classification system. In one aspect the system includes a digital ID generator and an ID database coupled to receive IDs from the ID generator. The system further includes a characteristic comparison routine identifying the file as having a characteristic based on ID appearance in the appearance database.
In a particular embodiment, the file is an e-mail file and the system utilizes a hashing process to produce digital IDs. The IDs are forwarded to a processor via a network. The processor performs the characterization and determination steps. The processor then replies to the generator to enable further processing of the email based on the characterization reply.
In a further aspect, the invention comprises a method for identifying a characteristic of a data file. The method comprises the steps of: generating a digital identifier for the data file and forwarding the identifier to a processing system; determining whether the forwarded identifier matches a characteristic of other identifiers; and processing the e-mail based on said step of determination.
In yet another aspect, the invention comprises a method for providing a service on the Internet, comprising: collecting data from a plurality of systems having a client agent on the Internet to a server having a database; characterizing the data received relative to information collected in the database; and transmitting a content identifier to the client agent. In this aspect, said step of collecting comprises collecting a digital identifier for a data file. In addition, said step of characterizing comprises: tracking the frequency of the collection of a particular identifier; characterizing the data file based on said frequency; storing the characterization; and comparing collected identifiers to the known characterization


REFERENCES:
patent: 5465353 (1995-11-01), Hull et al.
patent: 5515513 (1996-05-01), Metzger et al.
patent: 5619648 (1997-04-01), Canale et al.
patent: 5884033 (1999-03-01), Duvall et al.
patent: 5999932 (1999-12-01), Paul
patent: 6052709 (2000-04-01), Paul
patent: 6094487 (2000-07-01), Butler et al.
patent: 6144934 (2000-11-01), Stockwell et al.
patent: 6167457 (2000-12-01), Eidson et al.
patent: 6178417 (2001-01-01), Syeda-Mahmood
patent: 6189026 (2001-02-01), Birrell et al.
patent: 6195698 (2001-02-01), Lillibridge et al.
patent: 6199081 (2001-03-01), Meyerzon et al.
patent: 6199102 (2001-03-01), Cobb
patent: 6249805 (2001-06-01), Fleming, III
patent: 6310966 (2001-10-01), Dulude et al.
patent: 6321267 (2001-11-01), Donaldson
patent: 6330590 (2001-12-01), Cotten
Gary Boone “Concept features in Re:Agent, an intelligent email agent”, Autonomous Agents 1998, pp. 141-148.*
Robert J. Hall “How to avoid unwanted email”, ACM 1998, pp. 88-95.*
Cranor et al. “Spam!”, ACM 1998, pp. 74-83.*
Chang et al “Knowledge-based message management system”, ACM 1987, pp. 213-236.*
Ding et al “Centralized content-based web filtering and blocking: how far can it go?”, IEEE 1999, pp. 115-119.*
Distributed Checksum Clearinghouse, various such pages from www.rhyolite.com/anti-spam/dcc, Dec. 7, 2001.
Vipul's Razor, version such pages from razor.sourceforge.net printed Dec. 19, 2001.

Affiliated with

Also associated with

Rating

Distributed content identification system does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Distributed content identification system, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Distributed content identification system will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-2989953