Patent Application
20150042777
This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2013-163632, filed on Aug. 6, 2013, the entire contents of which are incorporated herein by reference.
The embodiments discussed herein are related to, for example, a perusing determination device, a perusing determination method, and a perusing determination program.
Up until recently, various pieces of information including character information have been created as digital data. For example, there has been increased an opportunity to read an electronic document such as an electronic book and a document file through a monitor. In addition, there also has been increased an opportunity to read on a monitor, for example, a document that is desired to be perused and includes a content to be grasped correctly such as a business e-mail, a procedure manual, and an instruction manual.
On the other hand, recently, a technology has been desired by which it may be automatically determined whether or not a user peruses an electronic document. For example, in Japanese Laid-open Patent Publication No. 2006-107048, a method of detecting order in which a user has referred to parts that are included in an electronic document, based on a direction of the user's gaze has been discussed.
In accordance with an aspect of the embodiments, a perusing determination device includes, a processor; and a memory which stores a plurality of instructions, which when executed by the processor, cause the processor to execute, detecting a movement direction of line-of-sight of a user who reads a document; counting a number of times in which the movement direction of the line-of-sight is reversed from a direction in which the document is to be read; and determining whether or not the user peruses the document, based on a counting result of the counting and a number of backward-directed eye movements that is to be satisfied at a time of perusing and is defined based on a feature of the document.
The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed.
These and/or other aspects and advantages will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawing of which:
An information processing system according to a first embodiment is described below with reference to
The information processing system 100 includes an information processing device 10 as a perusing determination device, a display 12, an input unit 14, and a line-of-sight detection device 16. The information processing device 10 displays a content and a document on the display 12 in response to an instruction from a user, and executes processing and the like of determining whether or not the user has perused the content and the document, by obtaining a detection result of the line-of-sight detection device 16. The detailed structure and processing of the information processing device 10 are described later.
The display 12 includes a liquid crystal display, and displays the content, the document, and the like in response to an instruction from the information processing device 10. In the embodiment, as an example, a case is described in which an electronic document having a layout as illustrated in
The input unit 14 includes a keyboard, a mouse, a touch-screen, and the like, and receives an input from the user.
The line-of-sight detection device 16 includes a near-infrared lighting (light emitting diode: LED) and a camera, and is a device that detects a line-of-sight direction in a noncontact manner by a corneal reflection method. The line-of-sight detection device 16 is provided in a part of or in the vicinity of the display 12, and may detect a position in the display 12 to which the line-of-sight of the user who sees the display 12 is directed. The line-of-sight detection device 16 may detect the line-of-sight direction of the user by a method other than the corneal reflection method.
In
In
The eye tracking unit 30 calculates time variation in a line-of-sight position that is detected in the line-of-sight detection device 16. That is, the eye tracking unit 30 detects a direction in which the line-of-sight of the user is moved on the electronic document (
The determination unit 32 determines whether or not the user has perused the electronic document, based on the counting result of the return movement counting unit 31. Here, in the embodiment, the determination unit 32 determines, through the line-of-sight of the user, whether or not the user has perused the electronic document, based on the number of return sweeps or the number of backward-directed eye movements that include regression and refixation and are specific movements of the line-of-sight when the user peruses a character string. The notification unit 34 notifies the user of the determination result of the determination unit 32.
Processing by the information processing device 10 according to the first embodiment is described in detail below with reference to the flowchart of
In the processing in
When “Yes” is determined in Step S10, in next Step S12, the eye tracking unit 30 sets a value t that indicates a serial number of a line-of-sight position at 1, and sets a value N that indicates the number of backward-directed eye movements at 0.
After that, in Step S14, the eye tracking unit 30 obtains a line-of-sight position Pt (xt, yt) (here, P1 (x1, y1)) on the screen of the display 12 from the line-of-sight detection device 16.
After that, in Step S16, the eye tracking unit 30 obtains a line-of-sight position Pt+1 (xt+1, yt+1) (here, P2 (x2, y2)) on the screen of the display 12 from the line-of-sight detection device 16. After Step S14, it is assumed that Step S16 is executed after a certain time period has elapsed (for example, a time period that is desired to complete return sweep by the user (about tens of milliseconds)). As the certain time period, generally, a frame rate of the camera in the line-of-sight detection device 16 may be employed.
After that, in Step S18, the eye tracking unit 30 calculates a positional relationship “dt=xt+1−xt” that is related to an X-axis direction (direction in which the document is to be read) of Pt (xt, yt) and Pt+1 (xt+1, yt+1). Here, the eye tracking unit 30 calculates a positional relationship “d1=P2 (x2, y2)−P1 (x1, y1)”. When the line-of-sight position moves forward during sentence reading, the positional relationship dt becomes a positive value. When the line-of-sight position moves backward during sentence reading (here, a backward-directed eye movement includes a regression, a refixation, and a return sweep), the positional relationship dt becomes a negative value. That is, the eye tracking unit 30 also detects a movement direction of the line-of-sight by calculating the positional relationship dt. In Step S18, a distance between two points for the X-axis direction is calculated as the positional relationship, but a vector or the like may be calculated instead of the distance.
After that, in Step S20, the return movement counting unit 31 determines whether or not the positional relationship dt is less than 0 (negative). When “No” is determined in Step S20, that is, when the line-of-sight position is moved in the direction in which the electronic document is to be read, the flow proceeds to Step S24. In addition, when “Yes” is determined in Step S20, that is, the movement direction of the line-of-sight position is reversed from the direction in which the electronic document is to be read, the flow proceeds to Step S22.
When the flow proceeds to Step S22, the return movement counting unit 31 increments the number of backward-directed eye movements N by 1 (N=N+1), and the flow proceeds to Step S24.
When the flow proceeds to Step S24, the return movement counting unit 31 determines whether or not the end button has been pressed. When “No” is determined in Step S24, the flow proceeds to Step S26. Similar to Step S10, in Step S24, it may be determined whether or not a further certain input is performed, or it may be determined whether or not information by which completion of reading of the document may be estimated is obtained. As the information by which completion of reading of the document may be estimated, for example, information that indicates that the line-of-sight position exists near the end position of the electronic document, information on the number of lines that are read by the user, which is estimated from the movement of the line-of-sight, and the like are assumed.
In Step S26, the eye tracking unit 30 increments the value t by 1 (t=t+1). After that, the eye tracking unit 30 and the return movement counting unit 31 count the number of backward-directed eye movements N by repeating the processing and determination in Steps S16 to S26. In addition, when “Yes” is determined in Step S24, the flow proceeds to Step S28.
When the flow proceeds to Step S28, the determination unit 32 determines whether or not “N” is a threshold value (L+1) or more. Here, the threshold value is set at “L+1”, because it may be estimated that the user peruses the document when the number of backward-directed eye movements is larger than the number of lines (number of line changes), that is, when a backward-directed eye movement expect return sweep is performed more than once. The threshold value may be “L+a” (“a” is an integer other than “1”). The value a may be defined based on a feature of the document such as the length of the whole electronic document, the number of characters for one line, and the number of lines. When “Yes” is determined in Step S28, the flow proceeds to Step S30, and when “No” is determined in Step S28, the flow proceeds to Step S32.
When the flow proceeds to Step S30, that is, when “N” is “L+1” or more, the determination unit 32 determines that the user has perused the electronic document. In addition, when the flow proceeds to Step S32, the determination unit 32 determines that the user has not perused the electronic document.
The determination result of the determination unit 32 is transmitted to the notification unit 34. The notification unit 34 may display the determination result on the display 12. Instead of the display, for example, voice may be output using a speaker that is not illustrated. In addition, the determination result of the determination unit 32 may be used for processing other than notification. For example, display control may be performed depending on the determination result of the determination unit 32 so that a next content or electronic document is not allowed to be displayed when the user has not perused the electronic document. In addition, for example, control may be performed depending on the determination result of the determination unit 32 so that a certain button is not allowed to be pressed. In addition, the determination results of the determination unit 32 may be collected, and statistics may be taken whether or not the user who uses the content peruses the electronic document.
As described above, in the first embodiment, the threshold value that is used in Step S28 indicates the number of backward-directed eye movements that is to be satisfied at the time of perusing, which is defined by the feature of the electronic document.
As described above in detail, in the first embodiment, the eye tracking unit 30 detects a movement direction of the line-of-sight of the user who reads the electronic document, and the return movement counting unit 31 counts the number of times in which the movement direction of the line-of-sight of the user is reversed from the direction in which the electronic document is to be read. In addition, the determination unit 32 determines whether or not the user has perused the electronic document, based on the counting result of the return movement counting unit 31 and the number (i.e., a second number) of backward-directed eye movements that is to be satisfied at the time of perusing, which is defined by the feature of the electronic document (in the first embodiment, the threshold value L+1 or more). As a result, in the first embodiment, determination may be performed in which whether or not a backward-directed eye movement occurs through the line-of-sight of the user is considered, so that it may be accurately determined whether or not the user has perused the electronic document.
In addition, in the first embodiment, the number of backward-directed eye movements that is to be satisfied at the time of perusing (threshold value) is a value that is defined based on the number of lines (L) that are included in the document, so that an appropriate threshold value may be set in which the number of return sweeps of the user is considered.
There is a device error in the line-of-sight detection device 16, and it is probable that a backward-directed eye movement is recognized erroneously due to the device error. Here, it is known that the erroneous recognition occurs in accordance with certain probability distribution independently of a read character string. Therefore, the return movement counting unit 31 may determine whether or not the backward-directed eye movement occurs, by considering a tendency of a document that is a determination target (occurrence probability of erroneous recognition for each time period and read position) in addition to a positional relationship between coordinates.
A second embodiment is described below with reference to
In
In the processing of
After that, in Steps S14 to S18, similar to the above-described processing in the first embodiment, the eye tracking unit 30 identifies line-of-sight positions Pt (xt, yt) and Pt+1 (xt+1, yt+1), and calculates a positional relationship dt (=xt+1−xt) that is related to the X-axis direction (direction in which the document is to be read).
After that, in Step S20′, the return movement counting unit 31 determines whether or not the positional relationship dt is larger than a first threshold value T1 and smaller than a second threshold value T2. Here, the first threshold value T1 is a negative value, and for example, may employ a value such as “(80% of the length of a line)×(−1)”. The second threshold value T2 is also a negative value, and may employ a value such as “(value that is equal to or more than an error range of the line-of-sight detection device 16 (error range that is related to the X-axis direction))×(−1)”. As the second threshold value T2, “(size for the X-axis direction of the certain number of characters of an electronic document (for example, two characters))×(−1)” or “(size for the X-axis direction of a range in which characters are allowed to be recognized once, which is estimated from a visual field characteristic (for example, a central visual field)/2)×(−1)” may be employed. The first threshold value T1 and the second threshold value T2 may be set based on a further reference. When a condition of “T1<dt<T2” is satisfied, it is indicated that a backward-directed eye movement occurs.
When “No” is determined in Step S20′, that is, when a backward-directed eye movement does not occur, the flow proceeds to Step S24, and when “Yes” is determined in Step S20′, the flow proceeds to Step S22′. When the flow proceeds to Step S22′, the return movement counting unit 31 increments the number of backward-directed eye movements M by 1 (M=M+1). In addition, when “No” is determined in Step S24, the flow returns to Step S16 through Step S26, and the processing and the determination in Steps S16 to S26 are repeated until “Yes” is determined in Step S24.
After that, when “Yes” is determined in Step S24, the flow proceeds to Step S28′, and the determination unit 32 determines whether “M” is 1 or more, that is, whether the number of backward-directed eye movements is 1 or more. When “Yes” is determined in Step S28′, the flow proceeds to Step S30, and the determination unit 32 determines that the user has perused the electronic document. In addition, when “No” is determined in Step S28′, the flow proceeds to Step S32, and the determination unit 32 determines that the user has not perused the electronic document.
As described above, in the second embodiment, a device error of the line-of-sight detection device 16 is considered, so that it may be further accurately determined whether or the user has perused the electronic document. For example, when the line-of-sight stops, there is a case in which it is determined that the line-of-sight moves in a direction opposite to the direction in which the electronic document is to be read due to the drive error of the line-of-sight detection device 16, but it may be accurately determined whether or not the user has perused the electronic document by removing such a device error.
When a device error of the line-of-sight detection device 16 is small, the return movement counting unit 31 may determine whether or not the positional relationship dt is larger than the first threshold value T1 and smaller than 0 in Step S20′.
A third embodiment is described below with reference to
In the processing of
When “No” is determined in Step S102, the flow proceeds to Step S26. When “Yes” is determined in Step S102, the flow proceeds to Step S104.
When the flow proceeds to Step S104, that is, when there has occurred the return sweep, the return movement counting unit 31 stores “N” in a reversal DB. Here, it is assumed that the reversal DB have a data structure as illustrated in
Returning to
In addition, when “Yes” is determined in Step S24, the flow proceeds to Step S108. When the flow proceeds to Step S108, the determination unit 32 determines whether or not a tendency of the reversal DB is similar to that of a reference reversal DB (
When “Yes” is determined in Step S108, the flow proceeds to Step S30, and the determination unit 32 determines that the user has perused the electronic document. In addition, when “No” is determined in Step S108, the flow proceeds to Step S32, and the determination unit 32 determines that the user has not perused the electronic document.
As described above, in the third embodiment, the number of backward-directed eye movements for each of the lines is considered, so that the determination unit 32 may perform highly accurate perusing determination so as to consider a feature of the electronic document. In addition, even when the user who skims the electronic document pretends to peruse the electronic document by performing the backward-directed eye movements on purpose, it may not be determined that the user operation corresponds to perusing of the electronic document, by setting, as a criterion, whether or not the number of backward-directed eye movements for each of the lines is similar to the reference that is defined beforehand (reference reversal DB).
In the above-described third embodiment, the determination unit 32 obtains a variance (relative frequency) of the number of backward-directed eye movements in the each of the lines, in each of the reversal DB and the reference reversal DB, and may determine whether or not the tendency of the reversal DB is similar to that of the reference reversal DB, based on a total value and an average value of differences of variances (relative frequency) for each of the lines.
In the above-described third embodiment, the number of backward-directed eye movements is counted for each of the lines, so that it may be determined whether or not the user is perusing the electronic document even in the middle of reading of the electronic document by the user. Thus, the notification unit 34 may notify the user that it is desirable that the user peruses the electronic document in the middle of reading of the electronic document by the user, based on the determination result.
A fourth embodiment is described below with reference to
After Step S318, by repeating the processing and determination in Steps S318 to S328, the return movement counting unit 31 counts the value Nb of the number of backward-directed eye movements in the second half of the electronic document (S312). In addition, when “Yes” is determined in Step S328, that is, when the end button is pressed by the user, the flow proceeds to Step S330.
When the flow proceeds to Step S330, the determination unit 32 determines whether or not the value Na is the first half threshold value “R+a” or more, and the value Nb is the second half threshold value “S+b” or more. When “Yes” is determined in Step S330, the flow proceeds to Step S332, and the determination unit 32 determines that the user has perused the electronic document. In addition, “No” is determined in Step S330, the flow proceeds to Step S334, and the determination unit 32 determines that the user has not perused the electronic document. The determination unit 32 may determine that the user has perused the electronic document when at least one of “Na≧R+a” and “Nb≧S+b” is satisfied in addition to a case in which both “Na≧R+a” and “Nb≧S+b” are satisfied.
As described above, in the fourth embodiment, the electronic document is segmented into the first half and the second half, and it is determined whether or not the user has perused the electronic document, based on whether or not the number of backward-directed eye movements in each of the areas satisfies a threshold value that is defined in each of the areas. Therefore, the perusing determination may be performed accurately as compared with a case in which the perusing determination is performed using one threshold value. For example, in a regular electronic document, in the second half, it is highly probable that a content that has been described in the first half is explained in detail, and a new content is less than the first half, so that it is conceivable that the number of backward-directed eye movements is reduced. Thus, by applying different threshold values to the first half and the second half of the electronic document, appropriate perusing determination may be performed in which features of the document in the first half and the second half are considered.
In the above-described fourth embodiment, the electronic document is segmented into the first half and the second half, but the embodiment is not limited to such an example. The electronic document may be segmented into three or more, and a threshold value of the number of backward-directed eye movements may be set for each of the areas.
In the above-described fourth embodiment, a device error of the line-of-sight detection device 16 is considered, and the intermediate coordinates may be defined within the range.
In the above-described fourth embodiment, the case is described in which the different threshold values are applied to the first half and the second half, but the embodiment is not limited to such a case. In the first half, processing that is similar to the processing in one of the first to third embodiments may be executed, and in the second half, processing that is similar to the processing in one of the first to third embodiments and different from the processing in the first half may be executed.
In each of the above-described embodiments, the number of characters in each of the lines may be considered. For example, from among the lines, there is a line having a small number of characters (line having the number of characters is smaller than the certain number of characters as illustrated by an arrow in
In each of the above-described embodiments, an occurrence tendency of a backward-directed eye movement may be predicted, and a threshold value may be determined based on whether what kind of document the user is familiar with in the past, or a degree of difficulty for the user of electronic document. In this case, a threshold value is defined for each user.
In each of the above-described embodiments, the case is described in which the start button and the end button are arranged as illustrated in FIG. 4, but the embodiment is not limited to such a case. For example, the start button and the end button may be arranged as illustrated in
In each of the above-described embodiments, the case is described in which the electronic document corresponds to horizontal writing, but the embodiment is not limited to such a case, and the electronic document may correspond to vertical writing. Even in the case of the vertical writing, it may be accurately determined whether or not the user has perused the electronic document by executing processing that is similar to the processing in each of the above-described embodiments.
The processing that is described in each of the above-described embodiments may be executed by a further information processing device (server or the like) that is connected to the information processing device 10 through a network.
The above-described processing function may be obtained by a computer. In that case, there is provided a program in which a processing content of a function that is to be included in a processing device is described. By executing the program through the computer, the above-described processing function is achieved on the computer. The program in which the processing content is described may be recorded to a computer readable recording medium (here, carrier waves are excluded).
When the program is distributed, for example, the program is sold in the form of a portable recording medium such as a digital versatile disc (DVD) and a compact disc read only memory (CD-ROM) to which the program is recorded. In addition, the program may be stored in a storage device of a server computer, and the program may be transferred from the server computer to a further computer through a network.
The computer that executes a program stores, for example, a program that is recorded to the portable recording medium or a program that is transferred from the server computer, in the storage device of the computer. In addition, the computer reads the program from the storage device, and executes processing in accordance with the program. The computer may read the program from the portable recording medium directly, and execute the processing in accordance with the program. In addition, the computer may execute the processing in accordance with the read program each time the program is transferred from the server computer.
The above-described embodiments are preferable examples. However, the embodiments are not limited such examples, and various modifications may be made without departing from the gist of the present disclosure.
All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2013-163632 | Aug 2013 | JP | national |
