Data processing: measuring – calibrating – or testing – Measurement system – Performance or efficiency evaluation
Reexamination Certificate
2003-02-10
2004-12-07
Barlow, John (Department: 2863)
Data processing: measuring, calibrating, or testing
Measurement system
Performance or efficiency evaluation
C702S179000
Reexamination Certificate
active
06829564
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an apparatus for supporting evaluation of usability related to an operation with respect to a device or instrument which receives a user input and presents at least visual information in accordance with a content of the operation. An example of such an instrument would be a device that includes a graphical user interface (GUI).
2. Description of the Related Art
Instruments which present visual information related to user operation in order to supply various information, such as an instrument state and predetermined calculation result, to a user have become common and widespread. Examples of such instruments commonly found in most offices include copying machines, computers, and air conditioners. For example, most modern copying machines include a panel for setting the number of copies to be made and various copying functions (double-sided copying and reduction/enlargement), and a user operates the instrument based on the supplied visual information. Here, the visual information presented is not limited to dynamic information displayed on a liquid crystal display (LCD) such as information indicating the number of copies requested, but includes printed and fixed information, such as, for example, numerals “1”, “2”, . . . printed on a ten key keypad disposed for setting the number of copies to be made in the copying machine.
As instrument functionality has grown more sophisticated, operations have increasingly become complicated. International efforts have been made to facilitate improvement of instruments by providing indexes for evaluating the usability of instruments, such as user-friendliness of using the instrument, effectiveness of use of the instrument, efficiency, and satisfaction given to a user of the instrument. For example, the ISO 9241-11 standard (Ergonomic requirements for office work with visual display terminals (VDTs)-Part 11: Guidance on usability) defines evaluation measures related to usability from the viewpoints of effectiveness, efficiency, and satisfaction, and device usability can be enhanced through such evaluation. Here, usability indicates the degree of effectiveness, efficiency, and user satisfaction in achieving a designated object under designated circumstances of a certain product by a designated user. The effectiveness indicates accuracy and completeness in achieving a designated result, efficiency measures the amount of resources consumed in association with the accuracy and completeness in achieving the target by the user, and satisfaction measures comfort and acceptability for a user who uses a computer system and the computer system, that is, the degree to which a user is free from frustration and is able to maintain a positive attitude toward product use.
Known methods of performing evaluation of a task using information from users who have completed the relevant task include questionnaires in which users independently read and answer questions and interviews where users are asked questions and provide answers to evaluators. With a questionnaire, although a specific event at a task achieving time or an entire impression at a task end time can be better understand, a relationship between the operation content and problem is often unclear, and it is often difficult to use the information to improve the device.
There are known apparatuses for recording a user operation log and diagrammatically displaying the log record to facilitate analysis (e.g., Japanese Patent Application Laid-Open No. 8-153022). Moreover, there are techniques for recording and reproducing the operation log between the system, which is an evaluation object, and the user and listing and displaying the state log of the system (e.g., Japanese Patent Application Laid-Open No. 2001-51876).
Furthermore, there is another related art system in which user testing is employed. In user testing, a test subject regarded as a user actually operates the instrument, a test subject which attempting to perform a task is observed by an observer, the observer presumes what the test subject is going to do, and the observer notes steps at which the usability drops.
For example, when it is observed that the test subject's eyeshot moves over an operation panel, it is presumed that the test subject is searching the panel for information or a means to input a desired command. When this takes an excessively long amount of time, it is presumed that the operation of the panel is not clear and the panel design is not efficient for operation, and therefore the usability of such a panel is low.
However, in these related-art methods of evaluating usability, an evaluator must refer to records of the operation log, state log of the system, and test subject's state to conduct the evaluation. Therefore, evaluator's arbitrary judgment is easily included, and the final results tend to depend on the individual knowledge of each evaluator.
When there are a plurality of evaluators, particularly when there are many evaluation objects, uniform evaluation is not easily obtained, and the criteria or components to be improved are not easily distinguished. Moreover, to obtain the questionnaire or interview information from the test subject, only the content that the test subject is aware of can be extracted. Therefore, there is a large possibility that a test subject will unconsciously edit their thoughts or comments in order to describe the state after the end of the operation. Furthermore, with the known usability evaluation methods it is not always possible to obtain indexes of the usability such as effectiveness, efficiency, and satisfaction, which conform with the ISO 9241-11 standard.
Additionally, even if such indexes as effectiveness, efficiency, and satisfaction are obtained, because the content or definition of the index differs for different types of business, it is difficult to standardize evaluation. Moreover, effectiveness, efficiency, and satisfaction can be defined in each situation of use of the evaluation object. Therefore, when effectiveness, efficiency, and satisfaction data are not obtained according to the same definitions among the objects to be compared/evaluated, the meaning of quantification and evaluation is weakened. Additionally, there has been a demand for an ability to evaluate individual components of a task to evaluate their negative effect on the overall usability so that the specific components can be improved. Therefore, the burden on evaluators is heavy, and extensive training of evaluators is necessary.
SUMMARY OF THE INVENTION
The present invention was developed in consideration of the above-described circumstances, and provides an apparatus for supporting evaluation of usability so that usability can be easily and efficiently evaluated in consideration of circumstances regarding individual businesses in a mode suitable for evaluation conforming to ISO standards.
According to one aspect of the present invention, there is provided an apparatus for supporting evaluation of usability in achieving a given task with respect to an instrument including a display unit for displaying information, the apparatus comprising means for acquiring information of an operation log which is related to an operation performed by a user and which includes at least one item of operation log data collected during performance of the task; and index evaluation means for generating at least one effectiveness index related to accuracy and completeness in achieving a target of the task, efficiency index related to a resource consumed in association with the accuracy and completeness in achieving the target of the task, and satisfaction index related to satisfaction of a task achiever in achieving the target of the task. The index evaluation means evaluates and generates the effectiveness, efficiency, and satisfaction indexes using a statistical calculation involving at least one data included in at least one of a plurality of measurement items including the operation log information.
Moreover, the apparatus ma
Hirose Yoshitsugu
Mihira Toshie
Ohyama Tsutomu
Sakai Katsura
Shibuta Kazuo
Barlow John
Fuji 'Xerox Co., Ltd.
Oliff & Berridg,e PLC
Washburn Douglas N
LandOfFree
Usability evaluation support apparatus does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Usability evaluation support apparatus, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Usability evaluation support apparatus will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3281786