Distributed crawling of hyperlinked documents

Details Distributed crawling of hyperlinked documents Distributed crawling of hyperlinked documents

2007-12-04

2007-12-04

Bashore, William (Department: 2176)

Data processing: presentation processing of document, operator i

Presentation processing of document

Layout

C715S252000

Reexamination Certificate

active

09638082

ABSTRACT:
Techniques for crawling hyperlinked documents are provided. Hyperlinked documents to be crawled are grouped by host and the host to be crawled next is selected according to a stall time of the host. The stall time can indicate the earliest time that the host should be crawled and the stall times can be a predetermined amount of time, vary by host and be adjusted according to actual retrieval times from the host.

REFERENCES:
patent: 5974455 (1999-10-01), Monier
patent: 6032196 (2000-02-01), Monier
patent: 6182085 (2001-01-01), Eichstaedt et al.
patent: 6263364 (2001-07-01), Najork et al.
patent: 6321265 (2001-11-01), Najork et al.
patent: 6351755 (2002-02-01), Najork et al.
patent: 6374260 (2002-04-01), Hoffert et al.
patent: 6377984 (2002-04-01), Najork et al.
patent: 6418453 (2002-07-01), Kraft et al.
patent: 6424966 (2002-07-01), Meyerzon et al.
patent: 2005/0165778 (2005-07-01), Obata et al.
Thomas, Mike, A Web Crawler in Perl, Linux Journal (via ACM), Aug. 1997, pp. 1-5.

Affiliated with

Also associated with

Rating

Distributed crawling of hyperlinked documents does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Distributed crawling of hyperlinked documents, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Distributed crawling of hyperlinked documents will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3870876