Data processing: measuring – calibrating – or testing – Measurement system – Performance or efficiency evaluation
Reexamination Certificate
2001-12-03
2004-09-14
Hoff, Marc S. (Department: 2857)
Data processing: measuring, calibrating, or testing
Measurement system
Performance or efficiency evaluation
C702S176000, C702S187000, C702S188000
Reexamination Certificate
active
06792393
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates, generally, to a system and method for diagnosing the operational behavior of a computer system, especially, but not only, those running Internet applications and telecommunication applications. In particular, the present invention relates to diagnosing computer system operational behavior in terms of both application-level operation and system-level operation.
BACKGROUND OF THE INVENTION
In many computer systems, pre-production testing of computer systems, including application-specific computer systems like web servers, is critical to avoiding unexpected and expensive operational problems once the computer systems are deployed and relied upon. General areas of concern include processing bottlenecks and resource use problems (for example, memory and CPU usage). In particular, systems with varying loads (i.e., levels of use) or long-term load carrying may be susceptible to operational problems at higher loads or after operation for long periods of time, although such problems may not be evident in small-scale pre-production testing. A particular, although not exclusive, example of this concept is recognizable in web servers, where the loads on a web server (for example, client-sent file requests) may vary widely according to the number of users accessing the server. Also, extended periods of high server loads may occur (for example, on an e-commerce web server during a peak shopping period).
Conventional computer system testing systems are known for assessing computer system operation. However, conventional testing systems usually perform only a few functions, and do not provide a comprehensive diagnostic package.
For example, load test tools such as LoadRunner® are commercially available from Mercury Interactive of Sunnyvale, Calif. The LoadRunner® test tool is software-implemented and provides a mechanism for running load testing, collecting results, and reporting the collected results. However, it does not select or otherwise help to select specific parametric information to be collected or specific tests to be run. The computer system parameters measured are usually predetermined and not user-selectable. Thus, a system engineer or the like must be able to recognize which test data are of significance.
Similarly, conventional software-implemented application data collection tools are available, for example, from Keynote Systems Inc. However, the Keynote tool is limited to testing/monitoring web sites by deploying agents worldwide that test the client-level experience with respect to a given web site. The Keynote tool only measures user-level response times and throughput, and user-level errors (such as http error codes). It does not monitor a computer system at the server level, nor does it provide any diagnostic functionality. A user must independently decide what tests should be run and must interpret the results.
SE Toolkit is a conventional freeware performance analysis package. SE Toolkit provides system-level diagnosis tools but does not analyze user-experience parameters at an application level. Its primary focus is the computer system—specifically, operating system resource parameters. Analysis of the system parameters is limited to flagging system parameters whose values exceed predetermined thresholds. Some problem-solving functionality is provided, but it is limited to relatively simple problems corresponding to measurement of a single parameter. SE Toolkit provides a language interpreter for creating or editing measurement rules.
Quest Software, Inc. of Irvine, Calif. offers a software package called Spotlight™ that graphically displays server processes and data flow in real time. However, a user must identify congestion and bottleneck problems and decide what action to take. Spotlight™ is application-specific, so a given version of Spotlight™ is limited to analyzing server activity when running a specific application.
Quest also offers a software package called Foglight™ for monitoring application-wise use of system resources. In a particular embodiment, specific software “cartridges” are provided for specific applications to be monitored. Thus, general monitoring of applications with the Quest cartridges is not possible.
SUMMARY OF THE INVENTION
In contrast to the foregoing, the present invention relates to a diagnostic system for a computer system and a method for diagnosing computer system operation. An integrated treatment of both application-level and system-level operation with respect to a plurality of operational parameters is provided. As a result, the present invention can, for example and without limitation, identify performance problems, identify causes of the performance problems, make recommendations for overcoming performance bottlenecks, and provide some form of an index or relative score indicative of each operational parameter that has been examined. The present invention also can analyze the scalability of a computer system.
The present invention uses one or more of load and no-load test runs, time dependent data (such as soak testing), and empirically obtained information (such as field performance data). The input data may be either numerical parametric information or it may be qualitative descriptions of operational behavior that may be internally reduced to quantitative values.
Generally, the present invention includes a method of analyzing the behavior of a computing system. The term “behavior” may sometimes be used interchangeably with the term “signature” herein. The method includes identifying an undesirable computing system behavior and identifying its cause. Thereafter, the method includes recommending at least one of an application-based solution and a system-based solution to remedy the undesirable computing system behavior.
By developing a systematic, reusable approach to analyzing performance and scalability, the present invention significantly reduces the amount of time and resources required to gather and analyze the necessary performance data. The invention can be used effectively for any transaction-based application or operations support system. Examples of transaction-based applications include but are not limited to electronic mail, web servers, instant messaging, directory engines, web search engines, and single sign-on engines. Examples of operations support system applications include but are not limited to provisioning, trouble management systems, data reporting systems, and database information retrieval. The invention does not require a user to know software details of the application. In fact, the internal details of software are often hidden from users, so the invention assesses performance and scalability through empirical methods. The invention is particularly useful for applications that involve database access or applications that are coded using C, C++, Java or Corba, although it is not limited in scope to these applications. These advanced technologies are often used to facilitate rapid development and maintainability, sometimes at the expense of performance and scalability. Ideally the invention should be utilized during the performance testing cycle (before deployment) but it can also be utilized by applications that are already in the field.
In an example of a method according to the present invention, analyzing computer system performance may include applying a test sequence to a computer system running an application, collecting test results from the test sequence that reflect computer system performance, and identifying a characteristic computer system behavior based on the collected test results. On the basis of the identified computer system behavior, the method calls for either matching the identified computer system behavior with a reference computer system behavior having a known cause or, if the identified computer system behavior does not match a reference computer system behavior, recommending further action to obtain additional information usable to refine the identification of the characteristic computer system behavior. A comput
Farel Richard Alan
Futumura Kenichi
Hosseini-Nasab Mehdi
Karasaridis Anestis
Leighton, III William J.
AT&T Corp.
Hoff Marc S.
Suarez Felix
LandOfFree
System and method for diagnosing computer system operational... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for diagnosing computer system operational..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for diagnosing computer system operational... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3213290