PERUSING DETERMINATION DEVICE PERUSING DETERMINATION METHOD

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2013-163644, filed on Aug. 6, 2013, the entire contents of which are incorporated herein by reference.

FIELD

The embodiments discussed herein are related to, for example, a perusing determination device, a perusing determination method, and a perusing determination program.

BACKGROUND

Up until recently, various pieces of information including character information have been created as digital data. For example, there has been increased an opportunity to read an electronic document such as an electronic book and a document file through a monitor. In addition, for example, there also has been increased an opportunity to read, on a monitor, a document that is desired to be perused and includes a content to be grasped correctly such as a business e-mail, a procedure manual, and an instruction manual.

On the other hand, recently, a technology has been desired by which it may be automatically determined whether or not a user peruses an electronic document. For example, in Japanese Laid-open Patent Publication No. 2006-107048, a method of detecting order in which a user has referred to parts that are included in an electronic document, based on a direction of the user's gaze has been discussed.

SUMMARY

In accordance with an aspect of the embodiments, a perusing determination device includes a processor; and a memory which stores a plurality of instructions, which when executed by the processor, cause the processor to execute, detecting a movement direction of a line-of-sight of a user reading a document; measuring time for the user to read each line of the document based on a detection result of the detecting; and determining whether the user has perused the document based on variation of time for reading the each line.

The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF DRAWINGS

These and/or other aspects and advantages will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawing of which:

FIG. 1 is a schematic diagram illustrating a structure of an information processing system according to an embodiment;

FIG. 2 is a diagram illustrating a hardware structure of an information processing device illustrated in FIG. 1;

FIG. 3 is a functional block diagram illustrating the information processing device illustrated in FIG. 1;

FIG. 4 is a diagram illustrating an example of an electronic document that is displayed on a display;

FIG. 5 is a flowchart illustrating an example of processing of an eye tracking unit and a reading time acquisition unit;

FIG. 6 is a diagram illustrating an example of a reading time variance DB;

FIG. 7 is a flowchart illustrating an example of processing of a tendency calculation unit;

FIG. 8A is a diagram illustrating an example of a specific word table;

FIG. 8B is a diagram illustrating an example of a specific word occurrence number DB;

FIG. 9 is a flowchart illustrating an example of processing of a determination unit;

FIG. 10A is a diagram illustrating an example of a relative frequency table of reading time;

FIG. 10B is a diagram illustrating an example of a relative frequency table of specific words; and

FIG. 11 is a diagram illustrating an example of a modification of the relative frequency table of specific words.

DESCRIPTION OF EMBODIMENTS

An information processing system according to an embodiment is described below with reference to FIGS. 1 to 6. In FIG. 1, a structure of an information processing system 100 according to the embodiment is schematically illustrated.

The information processing system 100 includes an information processing device 10 as a perusing determination device, a display 12, an input unit 14, and a line-of-sight detection device 16. The information processing device 10 displays a content and a document on the display 12 in response to an instruction from the user, and executes processing and the like of determining whether or not the user has perused the content and the document, by obtaining a detection result of the line-of-sight detection device 16. The detailed structure and processing of the information processing device 10 are described later.

The display 12 includes a liquid crystal display, and displays the content, the document, and the like in response to an instruction from the information processing device 10. In the embodiment, as an example, a case is described in which an electronic document having a layout as illustrated in FIG. 4 is displayed on the display 12. The electronic document in FIG. 4 is a document that includes L horizontal lines, and is, for example, a document such as a business e-mail, a procedure manual, an instruction manual, a consent agreement, or the like. In addition, it is assumed that a start button is arranged at the upper left of the electronic document, and an end button is arranged at the lower left of the electronic document. The display 12 may include a plurality of displays (multi-display).

The input unit 14 includes a keyboard, a mouse, a touch-screen, and the like, and receives an input from the user.

The line-of-sight detection device 16 includes a near-infrared lighting (light emitting diode (LED)) and a camera, and is a device that detects a line-of-sight direction in a noncontact manner by a corneal reflection method. The line-of-sight detection device 16 is provided in a part of or in the vicinity of the display 12, and may detect a position in the display 12 to which the line-of-sight of the user who sees the display 12 is directed. The line-of-sight detection device 16 may detect the line-of-sight direction of the user by a method other than the corneal reflection method.

In FIG. 2, a hardware structure of the information processing device 10 is illustrated. As illustrated in FIG. 2, the information processing device 10 includes a central processing unit (CPU) 90, a read-only memory (ROM) 92, a random access memory (RAM) 94, a storage unit (here, a hard disk drive (HDD)) 96, an input/output interface 97, and a portable storage medium drive 99, and each of the configuration units in the information processing device 10 is connected to a bus 98. In the information processing device 10, a program that includes a perusing determination program and is stored in the ROM 92 or the HDD 96, or a program that includes the perusing determination program and is read from a portable storage medium 91 by the portable storage medium drive 99 is executed by the CPU 90, so that a function of each unit in FIG. 3 is achieved.

In FIG. 3, a functional block diagram of the information processing device 10 is illustrated. As illustrated in FIG. 3, in the information processing device 10, the CPU 90 executes the program, so that functions as an eye tracking unit 30, a reading time acquisition unit 31, the tendency calculation unit 33, the determination unit 32, and the notification unit 34 are achieved. In this regard, a reading time variance DB 41, a specific word table 42, and a specific word occurrence number DB 43, which are stored in the HDD 96 and the like are also illustrated in FIG. 3.

The eye tracking unit 30 calculates time variation of a line-of-sight position, that is detected by the line-of-sight detection device 16. That is, the eye tracking unit 30 detects a direction in which the line-of-sight of the user moves on the electronic document (FIG. 4).

The reading time acquisition unit 31 measures a reading time that is a time period from when a return sweep of a user's line-of-sight has occurred to when the next return sweep of the user's line-of-sight occurred (a word viewing time to read one line) and then stores the measured reading time of each line into the reading time variance DB 41. Note that the reading time variance DB 41 has a data structure as illustrated in FIG. 6. Specifically, the reading time variance DB 41 has fields of a line number and a reading time (unit: second).

The tendency calculation unit 33 refers to the specific word table 42, and obtains how many specific words that are defined in the specific word table 42 (refer to FIG. 8A) are included in each line of the electronic document, and stores the acquisition result into the specific word occurrence number DB 43 (refer to FIG. 8B). In this regard, the specific word table 42 has characters including difficult words, which may change (or reduce) the movement speed of the user's line-of-sight. Specifically, the specific word table 42 stores a numeric character, a numerical expression, a bold type, underlined, meshed, and the like as a character type and a format, and stores difficult words as keywords. Also, the specific word occurrence number DB 43 has fields of the line number and the number of occurrences. In this regard, the number of occurrences of the specific words is a factor influencing reading time of each line.

The determination unit 32 refers to the reading time variance DB 41 and the specific word occurrence number DB 43, and determines whether the user has perused the electronic document or not. The notification unit 34 notifies the user of the determination result of the determination unit 32.

Processing by the information processing device 10 according to the present embodiment is described in detail below with reference to the flowchart of FIGS. 5, 7, and 9. As a precondition that the processing is started, it is assumed that the electronic document in FIG. 4 is displayed on the display 12, and the user sees the display 12. In this regard, in the present embodiment, it is assumed that the number of characters in each line is fixed in order to make it easy to understand.

Here, when a user reads an electronic document in order to understand the contents thereof, if the document includes difficult words, reading speed becomes slow compared with the case of a simple document. Accordingly, it is thought that variation (variance) occurs in reading time of each line depending on the existence or absence (the number of) difficult words in the document. On the other hand, when the user skims through an electronic document, reading speed of each line is generally the same, and substantially no variation occurs in reading time of each line. In the present embodiment, a determination is made of whether the user has perused the electronic document or not from such a characteristic of reading time.

[Processing of the Eye Tracking Unit 30 and the Reading Time Acquisition Unit 31]

FIG. 5 illustrates a flowchart of the processing of the eye tracking unit 30 and the reading time acquisition unit 31. In the processing in FIG. 5, first, in step S10, the eye tracking unit 30 waits until the user presses the start button through the input unit 14. In step S10, the eye tracking unit 30 may wait until the user performs a certain trigger operation instead of the pressing of the start button. As the certain trigger operation, there is an operation in which the user fixes the eyes on a certain position and an operation that is related on the line-of-sight such as wink. In addition, the eye tracking unit 30 may wait until utterance of certain voice by the user is detected using the microphone. The eye tracking unit 30 is started up after an input from the user through the input unit 14 is accepted, and may wait until a certain operation is detected in step S10. Alternatively, the eye tracking unit 30 may typically be running. When the eye tracking unit 30 is typically running, prediction of a line-of-sight position using previous line-of-sight information allows a case in which line-of-sight is not detected due to wink at a time of the trigger operation, and the like to be dealt with.

When “Yes” is determined in step S10, in next step S12, the eye tracking unit 30 sets a value t that indicates a serial number of a line-of-sight position at 1, and sets a value N that indicates the number of lines at 1, and starts the timer.

After that, in step S14, the eye tracking unit 30 obtains a line-of-sight position P_t(x_t, y_t) (here, P₁(x₁, y₁)) on the screen of the display 12 from the line-of-sight detection device 16.

After that, in step S16, the eye tracking unit 30 obtains a line-of-sight position P_t+1(x_t+1, y_t+1) (here, P₂(x₂, y₂)) on the screen of the display 12 from the line-of-sight detection device 16. After step S14, it is assumed that step S16 is executed after a certain time period has elapsed (for example, a time period that is desired to complete return sweep by the user (about tens of to hundreds of milliseconds)).

After that, in step S18, the eye tracking unit 30 calculates a positional relationship “d_t=x_t+1−x_t” that is related to an X-axis direction (direction in which the document is to be read) of P_t(x_t, y_t) and P_t+1(x_t+1, y_t+1). Here, the eye tracking unit 30 calculates a positional relationship “d₁=x₂−x₁”. When the line-of-sight position moves forward during sentence reading, the positional relationship d_tbecomes a positive value. When the line-of-sight position moves backward during sentence reading, the positional relationship d_tbecomes a negative value. That is, the eye tracking unit 30 also detects a movement direction of the line-of-sight by calculating the positional relationship d_t. In step S18, a distance between two points for the X-axis direction is calculated as the positional relationship, but a vector or the like may be calculated instead of the distance.

After that, in step S20, the reading time acquisition unit 31 determines whether or not the positional relationship d_tis less than −C. Here, −C means the distance of movement of the line-of-sight estimated to have performed a return sweep. When “NO” is determined in step S20, that is, when the user's line-of-sight has not performed a return sweep, the flow proceeds to step S24. On the other hand, when “YES” is determined in step S20, that is, when the user's line-of-sight has performed a return sweep, the flow proceeds to step S22.

If the flow proceeds to step S22, the reading time acquisition unit 31 records the reading time of the N-th line in the reading time variance DB 41 in FIG. 6, and the flow proceeds to step S24. Here, it is possible to obtain the reading time of the N-th line by subtracting the total of all the reading time (all the reading time from the first line to the (N−1)-th line) stored in the reading time variance DB 41 from the measurement time of the timer.

When the flow proceeds to step S24, the reading time acquisition unit 31 determines whether or not the end button has been pressed. When “No” is determined in step S24, the flow proceeds to step S26. Similar to step S10, in step S24, it may be determined whether or not a further predetermined input is performed, or it may be determined whether or not information by which completion of reading of the document may be estimated is obtained. As the information by which completion of reading of the document may be estimated, for example, information that indicates that the line-of-sight position exists near the end position of the electronic document, and information that the number of lines (N) that are read by the user has reached all the number of lines (L) of the electronic document, and the like are assumed.

In step S26, the eye tracking unit 30 increments the value t by 1 (t=t+1), and increments N by 1 (N=N+1). After that, the eye tracking unit 30 and the reading time acquisition unit 31 repeat the processing and determination in steps S16 to S26 so as to record the reading time of each line of the electronic document into the reading time variance DB 41. When “Yes” is determined in step S24, all the processing in FIG. 5 is terminated.

[Processing of the Tendency Calculation Unit 33]

FIG. 7 illustrates processing of the tendency calculation unit 33 by a flowchart. The processing in FIG. 7 is for example, the processing that is started at the timing when “Yes” is determined in step S10 in FIG. 5, and at the timing when “Yes” is determined in step S24, and the like.

In the processing in FIG. 7, first, in step S30, the tendency calculation unit 33 sets the value N indicating the number of lines to 1. Next, in step S32, the tendency calculation unit 33 obtains information of the N-th line (all the character strings). Next, in step S34, the tendency calculation unit 33 refers to the specific word table 42, and extracts specific words (words that changes (reduces) the movement speed of the user's line-of-sight) from all the character strings in the N-th line. In this regard, if a same specific word comes up two times or more in an electronic document, it is highly possible that the specific word in the second time and after will not change the movement speed of the user's line-of-sight. Accordingly, the tendency calculation unit 33 may store specific words that are existent in each line, and may not count the stored specific words at the second occurrence and after that.

In step S36, the tendency calculation unit 33 records the number of extracted words (number of occurrences) into the specific word occurrence number DB 43 as the N-th line data.

In step S38, the tendency calculation unit 33 determines whether or not N matches L, the number of all the lines of the electronic document. When “No” is determined, here, the flow proceeds to step S40, the tendency calculation unit 33 increments N by 1 (N=N+1), and the flow proceeds to step S32. After that, the processing and determination in steps S32 to S40 are repeated until “Yes” is determined in step S38. And when “Yes” is determined in step S38, that is, when the numbers of occurrences of the specific words in all the lines are stored in the specific word occurrence number DB 43, all the processing in FIG. 7 is terminated.

[Processing of the Determination Unit 32]

FIG. 9 illustrates a flowchart of processing of the determination unit 32. The processing in FIG. 9 is the processing to be started after both of the processing in FIG. 5 and FIG. 7 are terminated.

In the processing in FIG. 9, first, in step S50, the determination unit 32 generates a relative frequency table of the reading time. Specifically, as illustrated in FIG. 10A, the determination unit 32 generates the relative frequency table of the reading time, which has been produced by providing the reading time variance DB41 in FIG. 6 with a relative frequency field. In this regard, at the time of processing in step S50, it is assumed that the relative frequency fields are blank fields.

In step S52, the determination unit 32 generates the relative frequency table of specific words. Specifically, as illustrated in FIG. 10B, the determination unit 32 generates the relative frequency table of the specific words, which has been produced by providing the specific word occurrence number DB 43 in FIG. 8B with the relative frequency field. In this regard, at the time of processing in step S52, it is assumed that the relative frequency fields are blank fields. In this regard, the processing order in steps S50 and S52 may be the opposite.

In step S54, the determination unit 32 sets the N indicating the number of lines to 1, and sets the value K indicating the total value of the differences of the relative frequencies to 0.

In step S56, the determination unit 32 calculates a relative frequency of the reading time of the N-th line. Here, it is assumed that the relative frequency of the reading time in the N-th line is a ratio of the reading time of the N-th line to all the reading time. For example, it is assumed that when N=1 (in the case of the first line), “0.17” illustrated in FIG. 10A is calculated as a relative frequency.

In step S58, the determination unit 32 calculates a relative frequency of the number of occurrences of the specific words in the N-th line. Here, it is assumed that the relative frequency of the number of occurrences of the specific words in the N-th line is a ratio of the number of occurrences of the specific words in the N-th line to the number of occurrences in the entire document. For example, when N=1 (in the case of the first line), “0.17” illustrated in FIG. 10B has been calculated as the relative frequency. In this regard, the processing order in steps S56 and S58 may be the opposite. In the present embodiment, in order to make it possible to compare different scales of the reading time of each line, and the number of occurrences of the specific words in each line, normalization is performed using the relative frequency.

In step S60, the determination unit 32 calculates the difference d between the individual relative frequencies calculated in steps S56 and S58, respectively. When N=1 (in the case of the first line), d=0.17−0.17=0 is calculated.

In step S62, the determination unit 32 adds the difference d to K (K=K+d).

In step S64, the determination unit 32 determines whether N=L or not. That is, the determination unit 32 determines whether or not the processing in steps S56 to S64 has been performed until the L line. When “No” is determined here, the flow proceeds to step S65, N is incremented by 1 (N=N+1), and the processing returns to step S56. After that, the processing and determination in steps S56 to S65 are repeated. When “Yes” is determined in step S64, that is, in step S62, when the sum total of the differences d of the relative frequencies of all the lines is obtained, the flow proceeds to step S66.

When the flow proceeds to step S66, the determination unit 32 determines whether or not “K” is a predetermined threshold value or less. It is thought that the smaller the value of “K”, the more similar the distribution of the reading time in each line, and the distribution of the specific words in each line become. Accordingly, it is assumed that the threshold value here is about a value allowing a determination that variation of reading time and the variation of the number of occurrences of the specific words are similar. When “Yes” is determined in step S66, the flow proceeds to step S68, and else when “No” is determined, the flow proceeds to step S70.

When the flow proceeds to step S68, that is, when “K” is the threshold value or less, the determination unit 32 determines that the user has perused the electronic document. In addition, when the flow proceeds to step S70, the determination unit 32 determines that the user has not perused the electronic document.

The determination result of the determination unit 32 is transmitted to the notification unit 34. The notification unit 34 may display the determination result on the display 12. Instead of the display, for example, voice may be output using a speaker that is not illustrated. In addition, the determination result of the determination unit 32 may be used for processing other than notification. For example, display control may be performed depending on the determination result of the determination unit 32 so that a next content or electronic document is not allowed to be displayed when the user has not perused the electronic document. In addition, for example, control may be performed depending on the determination result of the determination unit 32 so that a certain button is not allowed to be pressed. In addition, the determination results of the determination unit 32 may be collected, and statistics may be taken whether or not the user who uses the content peruses the electronic document.

As described above in detail, by the present embodiment, the eye tracking unit 30 detects the movement direction of the line-of-sight of the user reading an electronic document. The reading time acquisition unit 31 obtains time to be spent by the user to read each line of the electronic document based on the detection result of the eye tracking unit 30. The determination unit 32 determines whether or not the user has perused the electronic document based on the total of the differences between relative frequencies of the reading time in individual lines, and the relative frequencies of the number of occurrences (factors influencing the reading time of each line) of the specific words, that is, a correlation between the variation in reading time of each line, and the variation of the number of occurrences of the specific words. Thereby, when the characteristic (the number of occurrences of the specific words in each line) of the electronic document is reflected on the reading time of each line, it is possible to determine that the user has perused the electronic document. Thus it is possible to determine the perusal by the user with high precision.

In the above-described embodiment, a description has been given of the case where the reading time of each line is determined by a time period produced by subtracting the total of the entire reading time (the entire reading time from the first line to the (N−1)-th line) stored in the reading time variance DB 41 from the measurement time of the timer. However, the present disclosure is not limited to this. For example, after “Yes” is determined in step S20 in FIG. 5, the timer may be stopped, and the timer may be started in step S26. In this case, the measurement time of the timer directly becomes the reading time of each line.

[Modification]

In the above-described embodiment, for the sake of convenience, a description has been given of the case where the number of characters in each line is fixed. However, in an electronic document, the number of characters in each line is often not fixed. In particular, the number of characters in the line of the closing paragraph sometimes includes only a few characters. In such a case, the tendency calculation unit 33 may create the relative frequency table of specific words in consideration of the number of characters in each line.

FIG. 11 illustrates an example of a relative frequency table of specific words according to the present variation. The relative frequency table of the specific words in FIG. 11 is produced by adding fields of the “number of characters” and the “relative frequency after correction” to the relative frequency table in FIG. 10B. The number of characters in each line is stored in the field of the “number of characters”. The product of the number of characters in each line divided by the maximum number of characters in one line (30 characters), and the relative frequency is stored in the field of the “after correction relative frequency”.

In this case, the determination unit 32 compares the relative frequency of the reading time in FIG. 10A, and the relative frequency of the specific words after correction in FIG. 11 for each line (calculates the difference d). If the total of the differences d is equal to or less than the threshold value, a determination ought to be made that the user has perused. In this manner, it becomes possible to determine with high precision whether the user has perused or not based on the number of specific words in each line and the number of characters.

In the above-described embodiment and modification, if variation of the reading time of each line is greater than the predetermined variation, a determination may be made that the user has perused. In this manner, it is possible to determine with precision that the user has not skimmed (there is a high possibility that the reading time of each line becomes substantially fixed) (has perused). In this regard, for variation of reading time of each line, for example, a value produced by totaling the differences between the average of the reading time of each line and the reading time of each line for all the lines may be employed. If that value is higher than the predetermined threshold value, the user may be determined to have perused.

In the above-described embodiment and modification, a description has been given of the case where the specific words reduce the movement speed of the user's line-of-sight. However, the present disclosure is not limited to this. The specific words may be words (for example, words capable of being skimmed through) that increase the movement speed of the user's line-of-sight. It is assumed that the words that increase the movement speed of the user's line-of-sight are ellipses “ . . . ”, words that occur repetitively, and the like. In the above-described embodiment, in the case of considering a word that increases the movement speed of the user's line-of-sight, the tendency calculation unit 33 ought to subtract the number of occurrences of the words that increase the movement speed of the user's line-of-sight from the number of occurrences of the words that reduce the movement speed of the user's line-of-sight, and may store the difference into the specific word occurrence number DB 43 in FIG. 8B.

In the above-described embodiment and modification, a description has been given of the case where as a method of determining whether there is a correlation between the reading time and the number of occurrences of specific words, a determination is made of whether the total of the differences of the relative frequencies of the individual lines is a threshold value or less. However, the present disclosure is not limited to this. That is, a determination may be made of whether there is a correlation between the reading time and the number of occurrences of specific words using a method of the other statistics processing, and the like.

In the above-described embodiment and modification, a description has been given of the case where the number of occurrences of specific words is used as a factor of influencing the reading time of each line. However, in combination with this, or in place of this, information of whether a same character type, such as a Japanese hiragana, a Chinese character, or the like continues for a predetermined number or more may be used as a factor influencing the reading time of each line.

In each of the above-described embodiments and modification, the case is described in which the electronic document corresponds to horizontal writing, but the embodiment is not limited to such a case, and the electronic document may correspond to vertical writing. Even in the case of the vertical writing, it may be accurately determined whether or not the user has perused the electronic document by executing processing that is similar to the processing in each of the above-described embodiments.

The processing that is described in each of the above-described embodiments and modification may be executed by a further information processing device (server or the like) that is connected to the information processing device 10 through a network.

The above-described processing function may be obtained by a computer. In that case, there is provided a program in which a processing content of a function that is to be included in a processing device is described. By executing the program through the computer, the above-described processing function is achieved on the computer. The program in which the processing content is described may be recorded to a computer readable recording medium (here, carrier waves are excluded).

When the program is distributed, for example, the program is sold in the form of a portable recording medium such as a digital versatile disc (DVD) and a compact disc read only memory (CD-ROM) to which the program is recorded. In addition, the program may be stored in a storage device of a server computer, and the program may be transferred from the server computer to a further computer through a network.

The computer that executes a program stores, for example, a program that is recorded to the portable recording medium or a program that is transferred from the server computer, in the storage device of the computer. In addition, the computer reads the program from the storage device, and executes processing in accordance with the program. The computer may read the program from the portable recording medium directly, and execute the processing in accordance with the program. In addition, the computer may execute the processing in accordance with the read program each time the program is transferred from the server computer.

All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.

PERUSING DETERMINATION DEVICE PERUSING DETERMINATION METHOD

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

US Classifications

International Classifications

Abstract

Description

Claims

Priority Claims (1)