Patent Application
20050268172
The present application claims priority upon Japanese Patent Application No. 2004-111139 filed on Apr. 5, 2004, which is herein incorporated by reference.
1. Field of the Invention
The present invention relates to a system, apparatus, method and program for evaluating usability to content.
2. Description of the Related Art
Recently, various types of information are distributed via the Internet and many companies compete fiercely for improving the quality of the web services to increase access rates or to acquire customers. Usability, i.e. an ease of use of a system is an important factor having influence on the quality of the web services. No matter how excellent the content provided there, if procedures for reaching to information are not intuitive or are complicated, or if very long time is needed for displaying the information, it is difficult to increase the access rate or to acquire customers.
Not to lag behind the current of the times, and to continuously maintain and improve the usability of the web services for attempting differentiation from other sites, it is essential to perform appropriate usability assessments. In order to perform the appropriate usability assessments, an arrangement is essential which collects information necessary for the assessments efficiently and which represents the usability performance in various and flexible forms using the collected information.
As a tool used for these usability assessments, for example, Japanese Patent Application Laid-open Publication No. 2001-51876 discloses a usability assessment apparatus for accurately logging system statuses as well as accurately reproducing the log to evaluate the ease of use of a system. Japanese Patent Application Laid-open Publication No. 8-161197 or HCI International 2003 Adjunct Proceedings, pp. 293-294, 2003 discloses a user interface assessment support apparatus and the like for storing a course of operations to a user interface shown on a window, determining the operation posing problems in the user interface based on the course of the stored operations or obtaining a degree of association among respective buttons on the user interface and displaying the results on the window. In accordance with this apparatus, workloads of assessors can be reduced by eliminating oversights of the assessors to prevent the assessors from failing to point out problems in the evaluation of the user interface. Also, incorporating the evaluation tool into a proxy server enables to capture page transition logs of users on the web, and complaints, ideas and desires harbored by the users when browsing the site via the Internet, without altering the site to be evaluated.
Also, Japanese Patent Application Laid-open Publication No. 2004-13242 discloses a method for supporting the usability assessments, wherein an assessment-target system does not have to be changed for the assessments by making a prompter-window correspond to each user interface of the assessment-target system involving transitions among multiple user-interface windows in advance, by displaying the corresponding prompter-windows when the user-interface windows of the assessment-target system are displayed, by storing user inputs to the prompter-windows and by sending the user data input from the prompter-windows to the assessment system. In accordance with this method, subjective assessment results from users can be obtained, along with log data for screen transitions operated by the users, in order to perform the usability assessments of the web sites on the Internet.
Japanese Patent Application Laid-open Publication No. 2003-316613 discloses a usability test system comprising simulation means for enabling simulated operations of an object on a terminal screen of a subject based on operations of the subject and information memory means for storing operation logs for each subject in association with execution of the simulation for provided questions, which enables analysis of operationality of the object and the like based on the operation logs obtained from the information memory means. In accordance with this system, a reliable usability test can be performed at low cost over a short amount of time.
By the way, in the usability assessments, it is important whether users themselves decide that the web service is easy to use, i.e. it is important to obtain users' subjective evaluation to usability. However, since the both techniques disclosed in Japanese Patent Application Laid-open Publication No. 2001-51876 and Japanese Patent Application Laid-open Publication No. 8-161197 use only the operation logs of users as targets of the analysis, users' subjective evaluation can not be known accurately. Also, since the technique disclosed in Japanese Patent Application Laid-open Publication No. 2003-316613 performs the usability assessments with operation logs, subjective evaluation can not be obtained.
The techniques disclosed in Japanese Patent Application Laid-open Publication No. 8-161197, Japanese Patent Application Laid-open Publication No. 2004-13242 and HCI International 2003 Adjunct Proceedings, pp. 293-294, 2003 obtain web page transition logs, and complaints, ideas and desires harbored by the users on the proxy server. However, since web sites increasingly provide dynamic contents operating only on the client side, it is difficult to accurately perform the usability assessments to these dynamic contents with the techniques sited in Japanese Patent Application Laid-open Publication No. 8-161197, Japanese Patent Application Laid-open Publication No. 2004-13242 and HCI International 2003 Adjunct Proceedings, pp. 293-294, 2003 which obtain data needed for the assessments on the server side. In the technique disclosed in Japanese Patent Application Laid-open Publication No. 2004-18242, evaluations input by users are output to a user answer DB, associating with a scenario which is definitions of query contents and answer methods to the assessment windows, and results of the assessments are output in the form associated with the scenario. However, in order to enable exact usability assessments to contents with expressions and functions increasingly broadened and complicated, arrangements are needed for providing assessment results expressed in more various and flexible forms.
A purpose of the present invention is to provide an information processing system, information processing apparatus, control method of the information processing system and a computer readable program which can provide usability assessment information in various forms by collecting usability assessment information efficiently and utilizing the collected information effectively.
In the present invention, a first information processing apparatus displays a window for the content on a display device depending on operations to an input device; the first information processing apparatus obtains feedback operation logs including assessment information for the window and date-time information when the assessment information has been input; the first information processing apparatus obtains operation logs including operation information to the input device and date-time information when the operation has been input; the first information processing apparatus obtains communication logs including date-time information when the window has been displayed and information for identifying the window; the first information processing apparatus sends to a second information processing apparatus the feedback operation logs, the operation logs and the communication logs; the second information processing apparatus receives from the first information processing apparatus the feedback operation logs, the operation logs and the communication logs; the second information processing apparatus correlates information included in at least two of the feedback operation logs, the operation logs and the communication logs in chronological order based on the date-time information included in each log; and usability assessment information is thus generated for the content.
According to the present invention, usability assessment information can be provided in various and flexible expression forms, since the feedback operation logs, the operation logs and the communication logs are correlated in chronological order based on the date-time information included in each log.
An embodiment of the present invention will now be described in detail along with the drawings.
The web server 20 is a computer for distributing content, which is to be a target of usability assessments, to apparatuses connected to the communication network 50. The terminal for assessor 10 is a computer operated by a user who performs the usability assessments. The user performs the usability assessments to the content such as a web page distributed from the web server 20 by inputting URL (Uniformed Resource Locator) to a web browser running on the terminal for assessor. The web browser incorporates a plug-in program achieving user interfaces for an operator of the terminal for assessor 10 inputting information on assessments of the content (hereinafter, referred to as assessment information). An install program for incorporating the plug-in program into the web browser is supplied to the terminal for assessor 10 by downloading from the server for assessments or the web server or from a portable recording medium. A plug-in ID is assigned to each plug-in program incorporated into the web browser, as an identifier. The plug-in ID is stored and managed by the terminal for assessor 10 to which the plug-in program is installed. The plug-in ID is utilized when the server for assessments 30 comprehends what terminal for assessor 10 runs the plug-in program performing a notification, for example. The assessment information obtained by the plug-in program is stored in a database 11 accessible to the terminal for assessor 10. In this embodiment, it is assumed that the database 11 is achieved as a function of the plug-in program.
The assessment information input with the terminal for assessor 10 and stored in the database 11 is transmitted to and accumulated in the server for assessments 20 via the communication network 50 or a portable recording medium. The assessment information transmitted to and accumulated in the server for assessments 20 is stored in a database 31 accessible to the server for assessments 20.
The terminal for assessor 10, the web server 20 and the server for assessments 30 do not necessarily have to be comprised of the entire structure shown as the above hardware and may be comprised at least of a structure needed for achieving each function. Specific examples of the computer used as the terminal for assessor 10 include personal computers, office computers, portable information terminals such as a PDA (Personal Data Assistant) and a CPU built-in cellular phone, and public broadcast receivers (analog broadcast receiver and digital broadcast receiver) equipped with the Internet connection function. Specific examples of the computer used as the web server 10 or the server for assessments 30 include personal computers, office computers, main frames and disk-array devices.
In
A feedback event handler 311 obtains information 353 showing operations associated with the content assessments out of input information notified from the input device 213 depending on input operations of the user, and stores into the database 11 the information generated based on the input information 353 as logs for operations associated with assessments (hereinafter, referred to as “feedback operation logs 372”). Also, the feedback event handler 311 notifies the operation event information acquisition component 312 and a content event information acquisition component 313 of an assessment-start instruction which is an instruction for starting acquisition of information associated with assessments and an assessment-end instruction which is an instruction for terminating the acquisition depending on the obtained input information (354, 355).
When the assessment-start instruction for the information associated with assessments is notified by the feedback event handler 311, the content event information acquisition component 313 sends a content acquisition request 358 to the web server 20 via the communication network 50 while communicating with the display controller 310 (356, 357), and receives content expression data 359 which are data for displaying the content (for example, HTML (Hypertext Markup Language) texts, XML (Extensible Markup Language) texts, programs executed in conjunction with or independently from these texts and other data associated with the content). Also, the content event information acquisition component 313 stores into the database 11 the logs for communications performed with the web server 20 on this occasion (hereinafter, referred to as “communication logs 373”).
The display controller 310 notifies the content event information acquisition component 313 of instruction information 356 such as URL of the content which is to be obtained, based on the input information 352 notified by the operation event information acquisition component 312. Also, the display controller 310 generates a window based on the content expression data 357 notified by the content event information acquisition component 313 and displays the window on the display devise 214 (360). A data transmitter 314 transmits to the server for assessments 30 the feedback operation logs 372, the operation logs 371 and the communication logs 373 stored in the database at the timing of the scheduled data and time or at appropriate timings when upload requests are received from the server for assessments 30.
Among the user interfaces provided in the upper area 401, an assessment-start button 410 is a button for indicating an intention to start assessments to the content. An assessment-end button 411 adjacent to the right thereof is a button for indicating an intention to end assessments to the content. When the assessment-start button 410 is clicked, the feedback event handler 311 is notified of clicking the button as input information from the input device 213, and the operation event information acquisition component 312 is notified of the assessment-start instruction for the information associated with the assessments described above. When the assessment-end button 411 is clicked, the feedback event handler 311 is notified of clicking the button as input information from the input device 213, and the operation event information acquisition component 312 is notified of the assessment-end instruction for the information associated with the assessments described above.
Among the user interfaces, buttons enabling the user to provide their impression 412a to 412d are buttons for expressing impressions felt about contents. 412a and 412b are buttons for expressing negative impressions, and 412c and 412d are buttons for expressing positive impressions. The user can easily assess contents by clicking one of multiple buttons enabling the user to provide their impression 412a to 412d provided correspondingly to types of the assessments in this way. When the user clicks the button corresponding to the impression felt about an image displayed in the lower area 402, the database 11 stores, as the feedback operation logs 372, the information indicating the type of the assessment comprehended based on information indicating which button is clicked out of buttons enabling the user to provide their impression 412a to 412d.
Among the user interfaces, a comment input field 414 is a field for inputting comments to contents. The comments input to the comment input field 414 are stored into the database 11 as the feedback operation logs 372 by clicking a registration button 415. A comment-only input button 413 adjacent to the right of the buttons enabling the user to provide their impression 412a to 412d is clicked when the user wants to input only the comment without performing the assessments using the buttons enabling the user to provide their impression 412a to 412d.
The aspects of the user interfaces described above are merely shown by way of an example. For example, the buttons enabling the user to provide their impression may be provided in different numbers from this example. Also, the display layout of the user interfaces is not limited to this, and the user interfaces may be provided in a right, left or bottom area of the web browser.
In step 706, the feedback event handler 311 generates a feedback event log ID which is an ID for uniquely identifying the feedback operation log 372 and the feedback operation log 372 stored in the database 11 is accompanied by the feedback event log ID. Also, in step 706, the feedback event handler 311 obtains data for generating a window currently displayed on the display device 214 by the functions of the display controller 310 (hereinafter, referred to as “hardcopy data”). The obtained hardcopy data are associated with the feedback event log ID as a file given a file name (hereinafter, referred to as “a picture image file”) and are stored into the database 11 as accompanying information of the feedback operation logs 372.
In step 702, if the feedback event generated is not an event for instructing start of assessments (step 702: NO), it is decided whether the event is an event for instructing end of assessments or not (step 707). If the feedback event is an event for instructing end of assessments (step 770: YES), the assessment flag is turned off (step 708), and the operation event information acquisition component 105 and the content event information acquisition component 106 is notified of the feedback event session ID and of an instruction indicating that the acquisition of the assessment information should be ended (step 709). Then, the processing proceeds to step 706.
In step 707, if the event is not an event for instructing end of assessments (step 770: NO), it is further decided whether the assessment flag is on or not (step 710). If the assessment flag is on (step 710: YES), the processing proceeds to step 706. If the assessment flag is off (step 710: NO), a message is displayed on the display device 214 in order to instruct the user to issue an assessment-start instruction (step 711) and the processing is returned to the reception waiting state for the feedback event (step 701).
In step 806, the operation event information acquisition component 312 checks whether the acquisition-end instruction for the assessment information is sent from the feedback event handler 311 or not (step 806). If the acquisition-end instruction exists (step 806: YES), the storage of the operation logs into the database 11 is stopped (step 807). If the acquisition-end instruction does not exist (step 806: NO), the processing proceeds to step 803 to wait for the generation of the operation event (step 803).
In step 906, the content event information acquisition component 313 checks whether the acquisition-end instruction for the assessment information is sent from the feedback event handler 311 or not (step 906). If the acquisition-end instruction exists (step 906: YES), the acquisition of the URL is stopped and the storage of the communication logs into the database 11 is stopped (step 907). If the acquisition-end instruction does not exist (step 906: NO), the processing proceeds to step 903 to wait for the generation of the operation event (step 903).
As described above, in the usability assessments system 1 of this embodiment, the assessment information such as the feedback operation logs 372, the operation logs 371 and the communication logs 373 is obtained by the functions of the plug-in program running on the client side. Therefore, accurate usability assessments can be performed, even to the dynamic contents constructed to achieve transitions of web pages on the client side.
Then, descriptions are made for the feedback operation logs 372, the operation logs 371 and the communication logs 373 which are the assessment information obtained and stored into the database as described above.
A feedback event log ID field 1001 lists the feedback event log IDs which are IDs for uniquely identifying the feedback operation logs 372. The feedback event log IDs are generated in step 706 of
An event generation time window field 1005 lists file names of files recording the hard copy data obtained in step 706 of
An operation log ID field 1101 lists the operation log IDs which are IDs for uniquely identifying the operation logs 371. The operation log IDs are generated in step 804 of
A content log ID field 1201 lists the content log IDs which are IDs for uniquely identifying the communication logs 373. The content log IDs are generated in step 905 of
For the assessment information described above, i.e. the feedback operation log table 1000, the operation log table 1100 and the communication log table 1200, details are transmitted to the server for assessments 30 by the date transmitter 314 via the communication network 50 accordingly. At the time of transmission, various pieces of information are also transmitted to the server for assessments 30, which are used for the usability assessments, such as the picture image files described above.
Based on the assessment information transmitted by the terminal for assessor 10 and received by the data receiver 391 as described above, the server for assessments 30 generates information helping the usability assessments (assessment result information) and generates various windows listing the information (hereinafter, referred to as “windows of assessment results”). The window of assessment result is generated by an assessment information generator 392 achieved by the CPU of the server for assessments 30 executing programs stored in the memory, and is displayed on the display device 214 of the server for assessments 30. If a printing apparatus such as a printer is connected to the server for assessments 30, the window of assessment result can be printed on pieces of paper and the like.
A transition order field 1401 displays IDs given in the order of the transitions of the windows. An operation date-time field 1402 displays details listed in the event generation time field 1004 of the feedback operation log table 1000. A window field 1403 displays images generated based on the files corresponding to the file names listed in the event generation window field 1005 of the feedback operation log table 1000. If the position information is listed in position information field 1008 of the feedback operation log table 1000, a mark 1410 made from a design corresponding to the assessment performed by the user is displayed at the position corresponding to the position information in the window displayed in the window field 1403. The assessment information generator 392 stores correspondences between image data prepared for respective types of the marks and the event types listed in the feedback event type field 1006 of
In accordance with the window of assessment result 1400 of
Details of a log ID field 1501 of the tabulated result table 1500 are the information respectively listed in the feedback event log ID field 1001 of the feedback operation log table 1000, the operation log ID field 1101 of the operation log table 1100 and the content log ID field 1201 of the communication log table 1200. Details of a feedback event session ID field 1502 are the information respectively listed in the feedback event session ID field 1002 of the feedback operation log table 1000, the feedback event session ID field 1102 of the operation log table 1100 and the feedback event session ID field 1202 of the communication log table 1200. Details of a plug-in ID field 1503 are the information respectively listed in the plug-in ID field 1003 of the feedback operation log table 1000, the plug-in ID field 1103 of the operation log table 1100 and the plug-in ID field 1203 of the communication log table 1200.
Details of a current URL field 1504 are the information listed in the current URL field 1205 of the communication log table 1200. Details of an event generation time field 1505 are the information respectively listed in the event generation date-time field 1004 of the feedback operation log table 1000, the event generation date-time field 1104 of the operation log table 1100 and the event generation date-time field 1204 of the communication log table 1200. Details of an event generation time window field 1506 are the information listed in the event generation time window field 1005 of the feedback operation log table 1000. Details of an operation object field 1507 are the information listed in the operation object field 1105 of the operation log table 1100. Details of an event field 1508 are the information listed in the operation object field 1105 of the operation log table 1100. Details of a feedback event type field 1509 are the information listed in the feedback event type field 1006 of the feedback operation log table 1000. Details of a comment detail field 1510 are the information listed in the comment detail field 1007 of the feedback operation log table 1000. Details of a position information field 1008 are the information listed in the position information field 1008 of the feedback operation log table 1000. Details of a registration button press-down time field 1512 are the information listed in the registration button press-down time field 1009 of the feedback operation log table 1000. Details of a changed URL field 1513 are the information listed in the communicated URL field 1206 of the communication log table 1200.
A window for assessment result 1600 shown in
In the window for assessment result 1600, the information relating to the usability assessments input by the user is displayed in the order of the transitions of windows displayed on the web browser of the terminal for assessor 10 due to operational inputs when the user browses the contents.
A transition order field 1601 displays IDs given in the order of the transitions of the windows. An operation date-time field 1602 lists details of the event generation date-time field 1505 of the tabulated result table 1500. A URL field 1603 lists details of the current URL field 1504 of the tabulated result table 1500. For example, in the tabulated result table 1500 of
A window field 1604 displays images generated based on the files corresponding to the file names listed as details of the event generation time window field 1506 of the tabulated result table 1500. An arrangement for displaying a mark 1610 listed and superimposed on the window is the same as the case of the mark 1410 described in
In accordance with the window for assessment result 1600 of
A button field displays the numbers of the assessments performed using each of the buttons enabling the user to provide their impression 412a to 412d and the comment-only input button 413 for the windows listed in the assessment-target URL and window field 1701. A comment display buttons 1720a to 1720e are buttons for displaying windows which list the comments input in the comment input field 414 or the comment input field 511 (comment list window 1800). An average assessment time field 1704 displays average times required to perform the assessments. The average times are obtained as average values of times between the clock times listed in the event generation time field 1505 and the clock times listed in the registration button press-down time field 1512.
The comment number field 1821 of the comment list 1820 in a lower section displays comment numbers which are IDs for identifying comments input when the assessments are performed using the button 1703b of
In accordance with the window for assessment result 1700 of
The above described usability assessment system separately obtains the assessment information such as the feedback operation logs, the operation logs and the communication logs, and correlates information included in the assessment information in chronological order based on the date-time information included in each log. In this way, as shown in
By the way, above descriptions of the embodiments are intended to facilitate understanding of the present invention and are not intended to limit the present invention. The present invention may be modified and altered without departing from the spirit of the present invention and the present invention includes equivalents thereof.
For example, the functions of the terminal for assessor 10 and the server for assessments 30 of the above embodiments can be achieved with the use of one computer. Although, in the above described embodiments, the server for assessments 30 has functions for centralizing the assessment information and displaying the assessment results, these functions can be provided in the terminal for assessor 10 and the web server 20. Also, the databases 11, 13 are not limited to the case of being provided in the above locations.
The contents to be assessed are not limited to the contents provided by the web server 20 described in the above embodiments, and the present invention can be applied to the case of performing the usability assessments to contents provided by various application systems operated with the use of client/server systems and peer-to-peer systems, for example. Also, the contents to be the targets of the usability assessments are not limited to the windows, and the present invention can be applied to the case when web applications are targets.
Although, in the above described embodiments, the URLs are used as the information for identifying the contents, the contents can be identified by a window name given to each of the contents. In this case, correspondences between the URLs and the window names must be managed.
Although, in the above described embodiments, four buttons enabling the user to provide their impression 412a to 412d are provided, the expression may be made by numeric values corresponding to levels. Also, the usability assessments may be performed effectively by preparing user interfaces with aspects preferred for assessing characteristics of the contents, such as buttons for assessing usefulness/unusefulness. Also, the present invention can be extensively applied to not only the case of performing the usability assessments of static aspects of the contents, such as designs of windows, but also the case of performing the usability assessments of dynamic aspects of the contents, such as response times.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2004-111139 | Apr 2004 | JP | national |
