Search engine with multiple crawlers sharing cookies

Details Search engine with multiple crawlers sharing cookies Search engine with multiple crawlers sharing cookies

2004-08-18

2009-06-09

Abdullahi, Salad (Department: 2451)

Electrical computers and digital processing systems: multicomput

Computer-to-computer session/connection establishing

C709S223000, C715S745000

Reexamination Certificate

active

07546370

ABSTRACT:
A web-crawler system includes a plurality of network crawlers configured to fetch documents from hosts on a network and a cookie database shared by the plurality of network crawlers. The cookie database stores cookies and associated information for use by the plurality of network crawlers. Each network crawler is configured to retrieve one or more cookies from the cookie database so as to enable access to documents on at least one of the hosts on the network. In some embodiments, each of the network crawlers may be configured to detect any of a plurality of predefined cookie errors associated with fetching a document. In some embodiments, each of the network crawlers may also be configured to detect when a cookie in the cookie database has expired and to obtain a replacement cookie.

REFERENCES:
patent: 6195622 (2001-02-01), Altschuler et al.
patent: 6263364 (2001-07-01), Najork et al.
patent: 6366947 (2002-04-01), Kavner
patent: 6418452 (2002-07-01), Kraft et al.
patent: 6499052 (2002-12-01), Hoang et al.
patent: 6665658 (2003-12-01), DaCosta et al.
patent: 6725269 (2004-04-01), Megiddo
patent: 2002/0023158 (2002-02-01), Polizzi et al.
patent: 2002/0057678 (2002-05-01), Jiang et al.
patent: 2002/0120685 (2002-08-01), Srivastava et al.
patent: 2002/0169818 (2002-11-01), Stewart et al.
patent: 2003/0037131 (2003-02-01), Verma
patent: 2004/0049673 (2004-03-01), Song et al.
patent: 2004/0128383 (2004-07-01), Hinton
patent: 2004/0205149 (2004-10-01), Dillon et al.
patent: 2005/0154887 (2005-07-01), Birk et al.
patent: 2005/0216845 (2005-09-01), Wiener
Raghavan et al. Crawling the Hidden Web. 2001. pp. 1-25.
Miller, Rob. WebSPHINX. 2002. pp. 1-8.
Alvarez et al. Client-Side Deep Web Data Extraction. 2004. pp. 1-4.
Kristol, D., et al., “HTTP State Management Mechanism, RCF 2109” Feb. 1997.
Kristol, D., et al., “HTTP State Management Mechanism RCF 2965,” Oct. 2000.

Affiliated with

Also associated with

Rating

Search engine with multiple crawlers sharing cookies does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Search engine with multiple crawlers sharing cookies, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Search engine with multiple crawlers sharing cookies will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-4112517