Data processing: measuring – calibrating – or testing – Measurement system – Performance or efficiency evaluation
Reexamination Certificate
2001-06-22
2004-07-13
Huff, Marc S. (Department: 2857)
Data processing: measuring, calibrating, or testing
Measurement system
Performance or efficiency evaluation
C709S224000
Reexamination Certificate
active
06763321
ABSTRACT:
BACKGROUND
1. Field of the Invention
The present invention relates to computer network servers. More specifically, the present invention relates to a method and an apparatus that facilitates measuring quality-of-service performance of a network server or collection of servers.
2. Related Art
Application availability is widely sought after as a requirement for applications delivered over networks. While users know what they want—continuous application access with predictable performance—it is often difficult to establish concrete measures that show whether the service providers charged with delivering the application over the network can meet these requirements. In an effort to quantify the availability of an application, system operators have established metrics that evaluate the end-to-end service response of an application delivered over a network.
In its classic form, availability is represented as a fraction of total time that a service needs to be available to perform its intended function. From a theoretical perspective, the asymptotic availability can be quantified as the ratio of the mean time between failures or interruptions (MTBF or MTBI) to the sum of the MTBF and the failure recovery time (also known as the MTTR, mean time to recovery). For example, a service that fails once every twenty minutes and takes one minute to recover can be described as having availability of ninety-five percent.
While availability metrics are useful, these metrics do not address the quality-of-service (QOS) from a user's perspective. A user is interested only in the response time, or wait time, between when a transaction is initiated and when the transaction is completed. For example, a user ordering a product over the Internet might click on a shopping cart to complete the transaction and then wait for the transaction to complete. Empirical evidence suggests that a typical user expects a response within about twelve seconds. A longer wait for a response may cause the user to abandon that site and use a competitor's site.
Although there are tools that may measure and log these QOS variables as a function of time, there are no tools that permit the diagnosis of interaction effects between and among the various QOS variables. A system analyst may be interested in the answers to questions such as: (1) If I have 10,000 email users this month, and I add 2,000 new email accounts next month, how will the email traffic impact my database users' wait times? (2) What impact do 8,000 animated browser banners have on download latency times for my 3,000 active file transfer protocol (FTP) users—and vice versa? (3) What impact do 4,000 active portable document format (PDF) file viewers have on transaction processing system (TPS) throughput? In general, what impact does the instantaneous demand for performance variable X produce in performance variable Y?
Attempting to measure the impact that one variable has on another variable is difficult, at best. One way of attempting to measure this interaction is to create a step increase in the number of transactions of one category and search for a change in the response, or wait, time in transactions of another category. The ability to find a change in the wait time in a second category of transaction requires the step increase in the first category of transaction to be relatively large. However, using a large step increase is undesirable because the large step increase may adversely affect the response time in the first variable also. Additionally, if the step increase is too large, the increase can cause the system to fail, or “crash.”
What is needed is a method and an apparatus to quantify the interaction of QOS variables as described above while minimizing the impact on the response times of the system.
SUMMARY
One embodiment of the present invention provides a system that facilitates measuring quality-of-service of a computer network server or collection of servers by determining the effect that transactions in one category of service have on transactions in other categories of service. The system operates by generating a varying pattern of synthetic transactions for one category of service and a fixed pattern of synthetic transactions for the other categories of service at a synthetic transaction generator. The system sends the varying pattern and the fixed pattern of synthetic transactions and receives responses to these patterns across a network or a direct coupling to the computer network server or collection of servers. The system then measures the response time for the varying pattern of synthetic transactions and the response time for the fixed pattern of synthetic transactions. Additionally, the system calculates the effect of transactions in one category of service on response times in the other category of service.
In one embodiment of the present invention, the system creates a table, indexed by time-of-measurement, of the response times for the categories of service.
In one embodiment of the present invention, the table includes a sliding-window of response times that includes the superposition of multiple systematically varying patterns of synthetic transactions.
In one embodiment of the present invention, the system calculates the effect of one category of service on the other categories of service by performing a bivariate statistical analysis on the table.
In one embodiment of the present invention, the bivariate statistical analysis includes a bivariate normalized cross power spectral density computation between the first response time and the second response time.
In one embodiment of the present invention, the result of the bivariate normalized cross power spectral density computation is analyzed using a Kolmogorov-Smirnov test of whiteness to determine dependencies between the first category of service and the second category of service.
In one embodiment of the present invention, the system creates multiple synthetic transaction generators at sites that are geographically separated from each other and geographically separated from the computer network server or collection of servers so that quality-of-service can be measured from widely separated sites.
In one embodiment of the present invention, the varying pattern of synthetic transactions includes a sinusoidally varying pattern of synthetic transactions.
In one embodiment of the present invention, the categories of service includes, but is not limited to, file transfers, database queries, email transactions, browser advertising banners, office applications, PDF file viewers, calendar updates, code executions, and file compressions.
REFERENCES:
patent: 5303166 (1994-04-01), Amalfitano et al.
patent: 5434848 (1995-07-01), Chimento et al.
patent: 5812780 (1998-09-01), Chen et al.
patent: 5819066 (1998-10-01), Bromberg et al.
patent: 6138157 (2000-10-01), Welter et al.
patent: 6157940 (2000-12-01), Marullo et al.
patent: 6167534 (2000-12-01), Straathof et al.
patent: 6317787 (2001-11-01), Boyd et al.
patent: 6324492 (2001-11-01), Rowe
patent: 6360332 (2002-03-01), Weinberg et al.
patent: 6477483 (2002-11-01), Scarlat et al.
patent: 6519714 (2003-02-01), Sweet et al.
Michael D. Beynon et al., Performance Evaluation of Client-Server Architectures for Large-Scale Image-Processing Applications. Published Dec. 1998.*
Publication, entitled “Sinusoidal Source Perturbation Experiments with a Breached Fuel Subassembly in the Experimental Breeder Reactor II,” to K.C. Gross and R.V. Strain, Nuclear Technology, vol. 98, pp 113-123, Apr. 1992.
Fishman David M.
Gross Kenny C.
Huff Marc S.
Miller Craig Steven
Park Vaughan & Fleming LLP
Sun Microsystems Inc.
LandOfFree
Method and apparatus to facilitate measurement of... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus to facilitate measurement of..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus to facilitate measurement of... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3250640