Data processing: database and file management or data structures – Database design – Data structure types
Reexamination Certificate
2000-08-01
2003-02-18
Choules, Jack (Department: 2171)
Data processing: database and file management or data structures
Database design
Data structure types
Reexamination Certificate
active
06523036
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to large-scale databases, and more particularly, to a database architecture having incremental scalability and that is adapted for use with Internet database systems.
2. Background Information
The amount of data generated by a typical Internet website is tremendous. There is a need for Internet applications that can store, manipulate, and retrieve large amounts of data. For example, a typical e-commerce website maintains information for each user, such as shipping and billing data, previous shopping experiences and category preferences. Popular websites may have millions of these data records. The explosive growth of Internet data is due to two primary factors. First, as the Internet expands, its reach becomes more pervasive, as more and more users are going online. Second, as Internet applications become more dynamic and personalized, more data are stored about each user. Therefore, data storage solutions become critical pieces of the Internet infrastructure requirements.
The term “netstore” as used herein is defined to be an Internet-scale data store that can handle both the traffic and capacity required by an Internet application. The netstore must have several capabilities. First, the typical number of total users that can access the netstore is extremely large (e.g., greater than 100 million users). Additionally, the typical number of concurrent users is large (e.g., 1 million users). Read operations to the netstore are more prevalent than write operations (e.g., a 10-1 read-to-write ratio for some Internet applications or even 100-1 for others). The netstore must be able to store a large amount of data and should be simple and flexible. Additionally, the data stored therein can be treated as a collection of picks that only has meaning to the particular Internet application.
Traditionally, data storage architectures for Internet applications, such as those that implement netstores, have been built upon relational and object-oriented database management systems (DBMS). These products have been developed primarily for the enterprise domain. However, it has been found that the data handling requirements of the Internet domain are significantly different than the requirements for a typical enterprise domain. Not only does the Internet domain place new demands on a netstore in terms of scalability, reliability and flexibility, the data model itself has changed. Most of these Internet applications require a very simple data model where the need to manage complex interrelationships in the data is deemphasized. Emphasis is instead placed on simplicity and flexibility of the data model. For instance, many Internet applications require the ability to read, write, or modify a single small data record individually.
Current DBMS products are not well suited for Internet applications because they have not been designed to address the distinct problem space presented by Internet applications. Consequently, solutions built using the enterprise DBMS products to address these Internet problems are costly to design, deploy and maintain.
Most of today's Internet sites that have read/write/modify storage requirements use relational database management systems (RDBMS). The reason why these sites choose RDBMS software is primarily one of convenience. There is an abundance of software tools that provide access to RDBMS products from web and application servers, thereby enabling sites to implement their netstores using off-the-shelf software.
In order to create a netstore with an RDBMS, the site must perform the following tasks:
(a) Design the database (i.e., tables, schema, relations, keys, stored procedures, etc.)
(b) Install, tune and maintain the database servers.
(c) Architect a scalable database system that is reliable, fault-tolerant and can handle the load and data required.
(d) Database-enable the web pages through a dynamic web server model. Typical options on Windows NT include: ASP/ADO (scripting) or ISAPI/ODBC (C/C++code). Typical options on Unix include: CGI-BIN/ODBC or NSAPI/ODBC/JDBC.
(e) Database-enable the application servers through custom code such as ODBC or JDBC.
Given the problem domain of enterprise-level database systems, limitations generally arise when they are used in a netstore implementation, since they are designed to be efficient at handling related data, and are not easily scalable. Key limitations with relational database systems used in a netstore environment include high maintenance costs, insufficient performance, poor scalability, and high implementation complexity.
It is therefore desired to provide a scheme that addresses the Internet application space directly through use of a specialized solution that provides a more optimized performance than conventional approaches, such as RDBMS. Preferably, the solution should be highly reliable, highly scaleable, and provide easy migration from existing products.
SUMMARY OF THE INVENTION
The present invention addresses the foregoing desires by providing an incrementally-scalable database system and method. The system architecture implements a netstore as a set of cooperating server machines. This set is divided into clusters, each of which consists of one or more server machines. All machines within a cluster are replicas of one another and store the same data records. The data is partitioned among the clusters, so that each data record in the netstore is stored in exactly one cluster.
This architecture allows for incremental scalability, load balancing, and reliability despite hardware or software failures. The system architecture enables database capacity to be scaled by adding resources, such as additional servers, without requiring that the system be taken offline. Such scaling includes both adding one or more computer servers to a given server cluster, which enables an increase in database read transaction throughput, and adding one or more server clusters to the system configuration, which provides for increased read and write transaction throughput.
The system also provides for load balancing read transactions across each server cluster, and load balancing write transactions across a plurality of server clusters. Read transactions can be served by different replicas at the same time, spreading out the load. For example, if there are 3 servers in a server cluster, approximately ⅓ of the requests will be routed to each machine, allowing for nearly 3 times the potential read transaction throughput of a single server. Since write requests are routed to a single cluster, adding clusters spreads out the write transaction load, with a similar effect on write throughput.
The system also provides for very high availability (HA) through its use of clustering. Because each of the machines in a server cluster is an identical replica of every other machine in the cluster, if that server fails, the problem is masked from the applications. The failed machine is removed from the system and the other replica servers in the cluster are available to satisfy requests for the failed server, without any impact to the application.
A system implementing the invention includes an application server layer, comprising one or more computers, serving as clients of a data storage layer, comprising of one or more server computers. The application server layer comprises compute servers that host an application program such as a web server. Also included is a scalable database server layer comprising of one or more server clusters, wherein each server cluster includes one or more database servers. Data is stored on the computer servers in the server clusters, wherein the data on each computer server in a given cluster is replicated. Under a typical configuration, the database(s) will comprise an RDBMS database such as a SQL-based database that comprises a plurality of record objects stored in tables defined by the database schema. The table data are partitioned into fragments and distributed across the server clusters such that each serv
Bobbitt Jared E.
Friedman Marc T.
Hickman Robert C.
Lau Patrick Wing Sang
Mullally Joseph P.
Blakely & Sokoloff, Taylor & Zafman
Choules Jack
Dantz Development Corporation
LandOfFree
Internet database system does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Internet database system, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Internet database system will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3140831