Method and apparatus for web crawler data collection

Details Method and apparatus for web crawler data collection Method and apparatus for web crawler data collection

2003-05-09

2008-11-18

Mofiz, Apu (Department: 2161)

Data processing: database and file management or data structures

Database design

Data structure types

C707S793000, C707S793000

Reexamination Certificate

active

07454410

ABSTRACT:
A Web crawler data collection method is provided for collecting information associated with a plurality of queries, which is used to calculate estimates of return probabilities, clicking probabilities and incorrect response probabilities. The estimated return probabilities relate to a probability that a search engine will return a particular Web page in a particular position of a particular query result page. The estimated clicking probabilities relate to a frequency with which a client selects a returned Web page in a particular position of a particular query result. The estimated incorrect response probabilities relate to the probability that a query to a stale version of a particular Web page yields an incorrect or vacuous response. Further, information may be collected regarding the characteristics and update time distributions of a plurality of Web pages.

REFERENCES:
Wolf, et al., “Optimal Crawling Strategies for Web Search Engines,” Proceedings of the 11th International Conference on World Wide Web, Session: Crawling, pp. 136-147, May 8, 2002.
The Eleventh International World Wide Web Conference, “Schedule-at-a-Glance,” printed on Oct. 7, 2005 from http://www.unizh.ch/home/mazzo/reports/www2002conf/program/schedule.html, 5 pages.
Lavrenko et al., “Relevance-Based Language Models,” SIGIR'01, Sep. 9-12, 2001, pp. 120-127.
Meretakis et al., “Extending Naive Bayes Classifiers Using Long Itemsets,” KDD-99, 1999, pp. 165-174.

Affiliated with

Also associated with

Rating

Method and apparatus for web crawler data collection does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Method and apparatus for web crawler data collection, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for web crawler data collection will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-4021464