Electrical computers and digital processing systems: multicomput – Computer network managing
Patent
1995-12-13
1999-10-26
Lee, Thomas C.
Electrical computers and digital processing systems: multicomput
Computer network managing
709217, 710 3, G06F 1202, G06F 1314
Patent
active
059744553
ABSTRACT:
A Web crawler system and method for quickly fetching and analyzing Web pages on the World Wide Web includes a hash table stored in random access memory (RAM) and a sequential Web information disk file. For every Web page known to the system, the Web crawler system stores an entry in the sequential disk file as well as a smaller entry in the hash table. The hash table entry includes a fingerprint value, a fetched flag that is set true only if the corresponding Web page has been successfully fetched, and a file location indicator that indicates where the corresponding entry is stored in the sequential disk file. Each sequential disk file entry includes the URL of a corresponding Web page, plus fetch status information concerning that Web page. All accesses to the Web information disk file are made sequentially via an input buffer such that a large number of entries from the sequential disk file are moved into the input buffer as single I/O operation. The sequential disk file is then accessed from the input buffer. Similarly, all new entries to be added to the sequential file are stored in an append buffer, and the contents of the append buffer are added to the end of the sequential whenever the append buffer is filled. In this way random access to the Web information disk file is eliminated, and latency caused by disk access limitations is minimized.
REFERENCES:
patent: 4323968 (1982-04-01), Capozzi
patent: 4847830 (1989-07-01), Momirov
patent: 5010344 (1991-04-01), Nagy
patent: 5357617 (1994-10-01), Davis et al.
patent: 5390318 (1995-02-01), Ramakrishnan et al.
patent: 5467264 (1995-11-01), Rauch et al.
patent: 5493676 (1996-02-01), Amundson
patent: 5708780 (1998-01-01), Levergood et al.
patent: 5712979 (1998-01-01), Graber et al.
Simpson et al., "The Searchers," Personal Computer Magazine, Jan. 1996, VNU Business Publications, UK, pp. 90-92, 97-98, 100.
Sha, V.T., "Cataloging Internet Resources: The Library Approach," Electronic Library, Oct. 1995, Learned Information, UK, vol. 13, No. 5, pp. 467-476.
Digital Equipment Corporation
Lee Thomas C.
Perveen Rehanba
LandOfFree
System for adding new entry to web page table upon receiving web does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System for adding new entry to web page table upon receiving web, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System for adding new entry to web page table upon receiving web will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-775790