Patent Application
20150363954
The present invention relates to apparatuses and methods for enabling improved reading in an electronic display.
Despite significant advances in digital technology, the illustration of textual information has not fundamentally changed. Texts are typically displayed in lines such that the reader's eye moves sequentially from word to word. With each eye movement (“saccade”), time is spent refixating the eye on the new word in order to recognize and process its meaning. The fixation takes about 240 milliseconds (“ms”) per word on average. Only 20% of the time for fixation is used for processing the content. It is also very common that a saccade does not reach the correct fixation point, so additional eye movement is required for reading a word.
One electronic display technique for reducing saccades is Rapid Serial Visual Presentation, hereinafter referred to as “RSVP.” RSVP was first introduced in the 1970s as a technique for presenting text one word at a time in a display. Many references since then have provided information on the use of RSVP in a variety of applications. Commercially available products based on RSVP include “Zap Reader” (www.zapreader.com/reader) and “Spreeder” (www.spreeder.com). Some prior methods exist for improving the effectiveness of an RSVP by varying the display time of a word in the display based on word length and word capitalization (see, U.S. Pat. No. 6,130,968 to McIan et al. (“McIan”)) and based on word frequency (see WO/37256 by Goldstein et al. (“Goldstein 2002”)). While these techniques are beneficial in improving comprehension of the displayed text, none of these teach how to minimize saccade movement during the presentation of a word or words in the RSVP display. Moreover, none of these techniques adequately take into account the nature of text written in Chinese characters.
RSVP has been used for electronically displaying text written in Chinese characters. However, the opportunities for optimizing RSVP display of Chinese and other languages based on Chinese characters have not yet been adequately explored.
Because Chinese and other languages based in whole or in part on Chinese characters are structured differently than English and other alphabet-based languages, different techniques are required for optimizing rapid serial visual presentation (“RSVP”) display of Chinese and related languages. For ease of description, embodiments of the present invention and the relevant principles are described in the context of Chinese only. However, many of the techniques described herein are relevant to other languages that use Chinese characters or characters derived from Chinese characters.
Commonly owned U.S. application Ser. No. 13/547,982 (now U.S. Pat. No. 8,903,174) the entire disclosure of which is herein incorporated by reference, describes certain techniques for optimizing RSVP display. As described in that application, conventional RSVP reduces saccades but does not eliminate them. Therefore, that application set forth techniques for an improved RSVP display that locates an optimal recognition position (“ORP”) of serially displayed words at a fixed location within the display. In order to prevent or minimize recovery saccades in an RSVP, it is preferable to display each word such that the optimal fixation position does not shift in the display. The focal point of the reader can then remain fixed on the optimal fixation position, which is a specific point in each word that is determined by the total number of characters or width of the word. This optimal recognition position, hereinafter referred to as the “ORP,” can be identified in the display such that the reader's eyes are directed to focus there as the words are serially presented. RSVP display that presents the ORP of each display element (e.g., word or phrase) at a fixed location on the display (i.e., the ORP of each serially presented display element is presented at the same location) allows text to be presented at a faster rate because no saccades occur during the presentation.
RSVP has been implemented at least in test setups for Chinese readers (see, e.g., Wang C X et al., “A comparison of Chinese reading parameters among rapid serial visual presentation, full screen and printed displays”). Conventional representation of Chinese text does not contain word separation by blanks Therefore, the words have to be separated from each other before being displayed in RSVP.
Some studies suggest that a preferred viewing position for a word in Chinese reading is left of middle (see, Xingshan Li, Pingping Liu, Keith Rayner: Eye movement guidance in Chinese reading: Is there a preferred viewing location?, Vision Research 51 (2011) 1146-1156)). For Chinese readers the spatial extent of visual processing during a fixation is 1 character to the left and 2-3 characters to the right of fixation (C.-H. Tsai & McConkie, 1995; Inhoff & Liu, 1997, 1998). More than 70% of all Chinese words are shorter than three characters. And less than 1% of all Chinese words are longer than four characters. Therefore, more than 99% of all Chinese words will fit into the spatial extent of visual processing during a single fixation.
Some embodiments of the invention comprise parsing a Chinese text into display elements for serial display, each display element comprising a short phrase (generally one or two words) that is five or fewer characters in length. In some embodiments, this includes parsing the text into sentences and words and then analyzing sequences of words as part of parsing the text into short phrases (generally one or two words in length) for creating display elements that are five or fewer Chinese characters in length.
Some embodiments of the invention comprise using a relative display time multiplier for each display element based at least in part on the display element's length. The relative time display multiplier is used in determining a display time for the display element (comprising a word or phrase). In some embodiments of the invention, blank elements are inserted between first and second sentences and displayed for a length of time that varies based on a word length of the first sentence. However, in a primary embodiment illustrated herein, blank elements are not displayed, but rather the last phrase of a sentence is displayed for a longer time than other phrases in the sentence. In some embodiments, a last display element of a sentence (which generally comprises a phrase and sentence-ending punctuation) is displayed for a longer time than other display elements in the sentence and the length of time varies if the word-length of the sentence is greater than a pre-determined number of words. In other embodiments, either a blank elements between sentences or a last phrase of a sentence is displayed for a fixed time relative to a selected reading speed (i.e., it does not vary based on sentence length).
In some embodiments of the present invention, the display time for a display element depends in part on the complexity and/or the use-frequency of its characters. In Chinese, the simplest character consists of a single stroke and the most complicated character includes more than 30 strokes (Togawa et al., 2000). Frequently used characters typically contain fewer strokes than do rarely-used characters. The average amount of strokes per character is approximately ten for the approximately 2,000 characters required for daily use. Thus, in some embodiments, characters containing more than ten strokes can be defined as being complex. In some embodiments, the use frequency of the word or words in a display element is used in determining a display time. In some embodiments, the use frequency of a phrase in a display element is used in determining a display time of the display element.
One embodiment of the present invention provides a method for serially displaying text on an electronic display comprising identifying an optimal recognition position for a plurality of display elements to be displayed and serially displaying the plurality of display elements (e.g., one or two word Chinese phrases comprising five or fewer Chinese characters) such that the optimal recognition position of each display element is displayed at a fixed display location on the electronic display. In one embodiment, the optimal recognition position is identified as a character in the phrase. In another embodiment, the optimal recognition position is identified as a proportionate position relative to the display width of the phrase. In some embodiments, visual aids are used to mark the fixed display location (e.g., hash marks) and/or an optimal recognition position within the phrase (e.g., different colored font).
Some embodiments comprise a computer program product including instructions for displaying text in accordance with principles of the present invention. Some embodiments comprise a computer program product including instructions for preparing and streaming text to be displayed in accordance with principles of the present invention. Some embodiments comprise an apparatus configured to carry out serial text display in accordance with principles of the present invention. Some embodiments comprise an apparatus (e.g., a computer system) configured to prepare and stream text to be displayed in accordance with principles of the present invention.
These and other embodiments are more fully described below.
The novel features of the invention are set forth in the appended claims. However, for purpose of explanation, several aspects of a particular embodiment of the invention are described by reference to the following figures.
a-d illustrate the display portion of the embodiment of
a and 5b illustrate display time multipliers for display elements (
a-6d are flow chart diagrams of text pre-processing in accordance with an embodiment of the present invention.
The following description is presented to enable any person skilled in the art to make and use the invention, and is provided in the context of particular applications and their requirements. Various modifications to the exemplary embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
User device 210 includes display 205, memory 207, and storage 277. Display 205 may be implemented as a touch screen display but is not limited to that. Display 205 further includes designated Rapid Serial Visual Presentation (“RSVP”) display area (DRDA) 200 on display 205. User device 210 also includes a processor and an input/output subsystem, which are not separately shown to avoid over-complicating the drawings.
Server computer 101 includes central processing unit (CPU) 110, storage 111 and memory 112. Server computer 101 may be any computing device capable of hosting computer product 211 for access by one or more client computers such as, for example, user device 210, over a network such as, for example, network 102 (e.g., the Internet). Server computer 101 communicates with one or more client computers via the Internet and may employ protocols such as the Internet protocol suite (TCP/IP), Hypertext Transfer Protocol (HTTP) or HTTPS, instant-messaging protocols, or other protocols.
Memory 112 and 207 may include any known computer memory device. Storage 111 and 277 may include any known computer storage device.
Although not illustrated, memory 112 and 207 and/or storage 111 and 277 may also include any data storage equipment accessible by the server computer 101 and user device 210, respectively, such as any memory that is removable or portable, (e.g., flash memory or external hard disk drives), or any data storage hosted by a third party (e.g., cloud storage), and is not limited thereto.
User device(s) 210 and server computer(s) 101 access and communicate via the network 102. Network 102 includes a wired or wireless connection, including a Wide Area Networks (WANs) and cellular networks or any other type of computer network used for communication between devices.
User device 210 may include any type of electronic device capable of controlling text display. Some examples include desktop computers and portable electronic devices such as mobile phones, smartphones, multi-media players, e-readers, tablet/touchpad, notebook, or laptop PCs, smart televisions, smart watches, head mounted displays, and other communication devices. In some implementations (e.g., a smart phone or e-reader), the display 205 may be packaged together with the rest of device 210. However, in other implementations, a separate display device (e.g., a monitor) maybe be attached to device 210. While the illustrated embodiment shows a graphical border around DRDA 200, DRDA 200 simply refers to a region (e.g., a window) on display 205 where text is serially presented in accordance with an embodiment of the present invention and in particular implementations, DRDA 200 may or may not be outlined by a graphical border.
In the primary embodiment described herein, computer program product 211 is loaded into memory 207 to configure device 210 in accordance with the present invention and the described processing of text for display is carried out on device 210. In such an embodiment, text data is loaded into memory for text processing and display processing by device 210 as will be further described herein. Text data loaded into memory for text processing and display processing may be retrieved from persistent storage on a user device such as storage 277 on device 210 and/or may be received from one or more server computers 101 through a connection to network 102 (e.g., the Internet). In an alternative embodiment, at least some processing/pre-processing of text data for display in accordance with the principles illustrated herein may be carried out by one or more remote computers such as server computers 101 and then sent to end user device 210 for display in DRDA 200 on display 205. In such an alternative, some or all of a computer program product such as computer program product 211 for implementing an embodiment of the present invention may reside on one or more computers such as server computers 101 that are remote from end user device 210. In some embodiments, the entire computer program product may be stored and executed on remote computers and the results presented within a browser application component (e.g. a media player application) of user device 210 (browser application and media player application not separately shown).
Although not illustrated, user device 210 also includes, as referenced above, a processor and an input and output subsystem. In some implementations (e.g., a smart phone or e-reader), user device 210 includes a wireless transceiver and one or more input interfaces including a touch enabled display, a trackball, keyboard, microphone, smart television controller, etc.
In an embodiment of the invention, text (which includes, for example, strings of characters—e.g., letters, Chinese characters, numbers, symbols, etc.—which constitute words, numeric figures, and combinations of both with punctuation marks and symbols—note that words may include words represented by any human language including, without limitation, words represented by roman characters, script elements, or pictographic characters—) is presented serially (for example, one word or one short phrase at a time) within DRDA 200. As referenced herein, a “display element” will refer to a group of text data that is displayed at one time within DRDA 200 and, in the context of Chinese characters, will generally include a short phrase. An embodiment displays display elements one after the other, i.e., serially.
In a conventional RSVP system, each word is centered in the display area, and the optimal fixation position shifts as words of differing lengths are sequentially displayed, resulting in saccade movements as the eyes shift to the optimal fixation position. The reader has to refocus on the display every time a new display element appears that is of a different length than the previous display element. The reader's eyes will move from one character to the next to find the optimal position, which is also referred to as a recovery saccade. In addition, when a longer display element follows a shorter one, the saccadic movement direction will be from right to left. When reading text in lines in a traditional paragraph display, most saccadic movement is from left to right so the reader is accustomed to this type of eye movement. Only occasionally, if the optimal fixation position is not found directly, the reader may have to move back from right to left. Thus conventional RSVP forces the reader to experience saccades which are not normal. Conventional RSVP approaches offer no solution to these problems. In order to prevent or minimize recovery saccades in an RSVP, it is preferable to display each word such that the optimal fixation position does not shift in the display. The focal point of the reader can then remain fixed on the optimal fixation position, which is a specific point in each word that is determined by the total number of characters or width of the word. This Optimal Recognition Position (ORP), can be identified in the display such that the reader's eyes are directed to focus there as the words are serially presented.
In one embodiment, configuring text content for RSVP display includes parsing the text into a plurality of display elements, inserting blank elements at the end of a sentence, and determining a multiplier for each display element that can be used, along with user selected settings and/or other display parameters, to determine a display time for each display element. While, in some embodiments, each element may be displayed for the same amount of time, it has been demonstrated empirically that a longer display time is beneficial for comprehension of longer words. It has also been demonstrated empirically that a longer pause between sentences is beneficial for comprehension of longer sentences. Further details of certain systems and methods for preparing and displaying text using RSVP are described in commonly-owned U.S. application Ser. No. 13/547,982 (issued as U.S. Pat. No. 8,903,174) referenced above.
Conventional representation of text in Chinese language, e.g. in printed form or in digital display does not present spaces between words. Readers often think in phrases instead of single words. Single Chinese words can often have several meanings; however, their meaning in the context of a phrase is easier to determine. Therefore, although some Chinese words may have clear meaning standing on their own, some words, particularly depending on the context, may be significantly easier to understand as part of a phrase. At the same time, certain phrases might be too long to avoid saccades during RSVP. Therefore, as further described in the context of
Most Chinese words are four characters or less in length. The visual span of Chinese readers is about four to five characters long, with one character to the left and two to three characters to the right. The ORP character position method for Chinese language centers the word or phrase to a specific character depending on the length of the displayed word or phrase as shown in TABLE 1. This definition of the ORP character position takes into account the asymmetric visual span of readers and the extend of the visual span for reading Chinese text. Punctuation marks which may occur after a word or phrase are not counted to the total number of characters for determination of the ORP Character Position.
Table I also shows the ORP Character Position Ratio for displaying Chinese text. The ORP Character Position Ratio value above is the proportion of the word that is to the left of the middle of the ORP character (e.g., if second character is centered at the fixed display location for the ORP, then 1.5 characters are to the left of that location) (note that in commonly owned U.S. application Ser. No. 13/547,982, issued as U.S. Pat. No. 8,903,174, a similar table value was calculated differently, based only on the number of whole characters to the left of the ORP character—therefore, in that application, an ORP Offset formula included accounting for the half-character width of the ORP character that was not accounted for in the table value. By contrast, the character position ratios in Table I above already account for the half-character width of an ORP character to the left of the fixed display location. Consistent with that, although not using the above ratio values in the primary embodiment, the ORP Offset formula presented herein in the context of
In other embodiments, the ORP is determined as a proportion of the display element's (word's or phrase's) pixel width. In most displays for Chinese texts all widths of characters are identical. Therefore, the pixel width of a display element can be calculated by multiplying the number of characters in the display element with the pixel width of one character. Based on this pixel width of a display element an offset is calculated beginning with the display element in the reading direction associated with the display element. If punctuation marks are connected with the word or phrase in the display, then this can be considered for calculating the offset. Punctuation marks in Chinese texts often do not fill out the entire character width. Therefore, it is possible to only consider the visible part of the character width for calculating the display elements' offset. A detailed description of the determination of the ORP by proportional offset is shown in
a shows a basic RSVP display 200 in accordance with the first embodiment of the present invention. For displaying Chinese text, the RSVP display 200 can accommodate text of up to eight characters in length, although it is preferred to limit the display to five characters for improved comprehension. Some additional room is preferred in any event because words or phrases can be displayed together with punctuation marks to the left and/or to the right of the word or phrase. In this embodiment, fixed display location 201 is positioned within the display 200 to accommodate two characters (plus any left side punctuation marks) to its left and can be identified with hash marks 202. Other identifiers such as circles, colors or textures may be used.
b shows how a circulating arrow 203 can be used to direct the user's eye to focus at fixed display location 201 before the words are serially presented. A countdown indicator 204 may also be utilized to indicate the amount of time before the words begin to be serially presented.
c illustrates positioning of the word “Hs” in RSVP display 200. When the serial presentation begins, the first word or phrase is displayed in the RSVP display 200 such that ORP character 205 is positioned at fixed display location 201. In the example in ” is positioned at fixed display location 201.
”. Based on TABLE I, the second character “
” is the ORP character 205 and is displayed at fixed display location 201.
A colored font, such as a red font, may be utilized to emphasize the ORP character 205, as shown in
Step 405 also calculates parameters for the display time of each display element. While, in alternative embodiments, it is possible to display each element for the same amount of time, it has been demonstrated empirically that a longer display time is beneficial for comprehension of longer words. It has also been demonstrated empirically that a longer pause between sentences (than between words) is beneficial for comprehension of longer sentences.
a provides a set of recommended values for a text element display multiplier 502 as a function of the text element character length 501 for application to Chinese language. Testing done by the inventor with native Chinese speakers has shown use of these values provides benefits to reading comprehension and therefore these values are provided for use in one embodiment of the invention. However, further empirical testing can be done to refine/modify these values and different values might be used in alternative embodiments without necessarily departing from the spirit and scope of the present invention.
b provides a set of recommended values for display multiplier 504 for a display element at the end of a sentence (e.g., last word or phrase of a sentence in combination with a punctuation mark such as a period, exclamation point, or question mark) as a function of the sentence length 503. Tests by the inventor using Chinese native speakers have shown that, for Chinese language, an increased display time of the last word (with the punctuation mark) in the sentence can aid in comprehension for some readers. This increased display time of the last word of the sentence is dependent on the amount of words in the sentence. The value of 2.1 for sentences up to 11 words for the display element multiplier works well. For sentences of 12 to 22 words a Display Multiplier of 3.2 is applied. And for sentences with 23 or more words a value of 4.3 is applied. It is therefore used in one embodiment of the present invention. However, further empirical testing can be done to refine/modify this value and different values might be used in alternative embodiments without necessarily departing from the spirit and scope of the present invention.
In the primary embodiment described above, a pause at the end of a sentence is achieved by displaying the last phrase (together with the punctuation mark) for a longer time based on a display multiplier. In that embodiment, no blank element is displayed prior to displaying the first phrase of the next sentence. However, in alternative embodiments, a pause is achieved by inserting and displaying a blank element between sentences. In such alternatives, multipliers such as multiplier 504 can be applied to vary the display time of the blank element based on the length of the sentences. Tests by the inventor using Chinese native speakers have shown that, for Chinese language, a short pause between sentence sentences can aid in comprehension for some readers.
Returning to the description of
a is a flow chart showing method 600 which illustrates further details of processing carried by computer application 300 of the embodiment of
Step 604 selects the sentence to be processed by step 605 (which is either the first sentence on an initial processing loop, or a next sentence in response to step 649). For that sentence, step 605 splits it into words. Step 606 arranges the words of the current sentence into phrases (see
Chinese phrases generally comprise either a single word or two words that together have a clear meaning. In some instances, a phrase may comprise more than two words, however, Chinese phrases, for purposes of embodiments of the present invention, usually include only one or two words. “Phrase” as used herein may therefore refer to either single words that have clear meaning used alone or small groups of words (usually no more than two) that, taken together, have a clear meaning.
Step 607 selects the phrase to be processed by step 608 (which is either the first phrase on an initial processing loop for a particular sentence, or a next phrase in response to step 642). For that phrase, step 608 calculates the phrase length (“pl”) with punctuation trimmed from that phrase. In this embodiment, pl is calculated without including punctuation for purposes of calculating a display multiplier and for determining an optimal recognition position character. However, alternative embodiments may utilize punctuation (or certain types of punctuation) for calculating certain display parameters, particularly to the extent particular empirical results dictate that factoring in punctuation can benefit reading results. For example,
Continuing with the description of
d shows the determination of the display multiplier value for a sentence-ending phrase. Step 651 determines if the current sentence consists of more than 22 phrases. If the answer is yes, then the display multiplier m is set to 4.3 at step 652) and the process continues to step 607 of
Continuing with the description of
Once m is set in block 611 (steps 612 to 617), block 618 then tests to determine if characters in the phrase are complex. Empirical studies have shown that readers of Chinese texts may focus longer on complex characters than on non-complex ones. Step 619 selects the first character of the current phrase and initiates the variable “ncc,” which counts the number of complex characters per phrase. Step 620 then determines if the character is complex. This can be done by a database query. In such a database, Chinese characters are designated as being complex or not. In one embodiment, this designation is based on the amount of included strokes per character. In one example, characters with more than 10 strokes are designated as complex. However, in alternative embodiments, different stroke criteria might be used to designate complexity. If the answer of step 620 is yes, then the variable ncc is increased by one and step 622 is executed. If the answer to step 620 is no, then step 622 is executed. Step 622 determines if the current character of the phrase is the last character of the phrase. If the answer is no, then step 623 selects the next character of the phrase. If the answer in step 622 is yes, then block 624 is executed.
Block 624 determines the number of infrequently used characters in a phrase. Step 625 selects the first character in the current phrase and initiates the variable nfc. This variable is used for specifying the number of not frequently used characters in a phrase. Step 626 then determines if the character is frequently used. This can be done by a database query. Such a database lists Chinese characters with their frequency of use. In one embodiment, the database lists use-frequency of the characters in general, without respect to the particular type of text. In another embodiment, the database lists use frequency of the characters in different specific types of text. This can allow refining the use-frequency determination based on the type of text that is being presented to the user. If the answer in step 626 is no, then step 627 increments the variable nfc by one before the method proceeds to step 628. If the answer to step 626 is yes, then the method proceeds step 628 directly. Step 628 determines if the current character is the last character of the phrase. If the answer is no, then step 629 selects the next character of the phrase. If the answer in step 628 is yes, then block 630 is executed.
Block 630 determines the number of infrequently used words in a phrase. Step 631 selects the first word in the current phrase and initiates the variable wf. This variable is used for specifying the number of words in a phrase that are used infrequently. Step 632 then determines if the word is frequently used. This can be done by a database query. Such a database lists Chinese words with their frequency of use. One example of such a database is described in: “SUBTLEX-CH: Chinese Word and Character Frequencies Based on Film Subtitles” by Qing Cai and Marc Brysbaert, available at: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0010729. The underlying data is available at http://crr.ugent.be/programs-data/subtitle-frequencies/subtlex-ch.
In one embodiment, the database lists use frequency of the words in general, without respect to the particular type of text. In another embodiment, the database lists use frequency of the words in different specific types of text. This can allow refining the use-frequency determination based on the type of text that is being presented to the user. If the answer of step 632 is no, then step 633 increments the variable wf by one and the method proceeds to step 634. If the answer to step 632 is yes, then the method proceeds directly to step 634. Step 634 determines if the current word of the phrase is the last word of the phrase. If the answer is no, then step 635 selects the next word of the phrase. If the answer in step 634 is yes, then step 636 is executed.
Step 636 determines if the current phrase is frequently used. This can be done by a database query. Such a database lists Chinese phrases with their frequency of use. In one embodiment, the database lists use frequency of the phrases in general, without respect to the particular type of text. In another embodiment, the database lists use frequency of the words in different specific types of text. This can allow refining the use-frequency determination based on the type of text that is being presented to the user. In the primary embodiment described herein, character complexity, word use-frequency and phrase use-frequency have simple binary designations (e.g., complex or not complex and frequently used or not-frequently used) and those results are used in a formula for conditionally modifying a display time based on character complexity and word and/or phrase use frequency. The inventors' research has shown that these binary designations are sufficient for providing beneficial effects on reading comprehension. However, alternative embodiments might rely on a greater number of gradations in determination of character complexity and word and phrase use-frequency and apply those determinations in a more complex formula than is illustrated in the primary embodiment. Some variations on the applied formula are further described below in the context of describing step 639.
If the result of step 636 answer is yes, then step 637 sets the variable pf to zero and step 639 is executed. If the answer is no, then step 638 sets variable pf to one and step 639 is executed.
Step 639 determines the display time multiplier (“dtm”). In one embodiment, this is done by calculating a “dtm_raw” value as dtm_raw=m+0.3(ncc+nfc+wf+pf) and then applying the following logic to determine a value for dtm:
If dtm_raw<=2: dtm=dtm_raw
if dtm_raw>2: dtm=2
Many variations on the above formula for dtm_raw are possible without departing from the spirit and scope of the present invention. For example, in alternative embodiments, a factor other than 0.3 can be used for determining dtm_raw. Also, in other alternatives, the variables ncc, nfc, wf and pf can have individual weightings that differ from each other and from 0.3. In other alternatives, only one of the variables ncc and nfc can be applied for computing the value dtm. Such an alternative recognizes that the complexity and the frequency of characters in Chinese language have a significant correlation (according to Togawa et al., 2000).
As referenced above, assessing the word frequency wf and the phrase frequency pf can be based on general statistics for Chinese texts or for sub-categories of Chinese text types. Also, the frequency can be based on customized statistics for the specific person reading the text. It's possible that a word or a phrase appears frequently for one reader, because of the texts that reader is consuming. The same word or phrase can appear rarely for another user because that reader reads different texts than the first reader. The different frequency of words or phrases can be used to determine customized display times of the word or phrase.
Step 640 saves the current phrase and its display time multiplier dtm (calculated in step 639) to the array “Phrases” (which collects the data for all phrases in the text). Step 641 increases the counter “pit,” which indicates the number of phrases in the text. Step 641 also increases the value of rtd by the amount of the value of dtm as determined in step 639. Step 642 then goes to the next phrase in the text.
If the answer in step 609 is yes (the current display element is a blank element, which follows the end of a sentence), then step 644 sets the display time multiplier (dtm) for the blank element to a fixed value, in this example, 0. Empirical studies by the inventors have shown that not showing a blank element after the sentence end works well. Therefore, in the primary embodiment, a pause between sentences is achieved by increasing the display time of the sentence's last phrase as discussed above. However, in alternative embodiments, as previously discussed, a blank element can be displayed to add a pause between sentences. Therefore, in alternative embodiments, a dtm for a blank element might be greater than zero. Step 645 saves the blank element and the associated display time multiplier to the array Phrases (note that, in this case, the value of the variable “Phrase” is simply the blank element). Step 646 determines if the end of the text is reached. If the answer is no, then step 649 goes to the next sentence in the text. If the answer to step 646 is yes, then step 647 saves the value of variable rtd and Step 644 ends the text processing.
Note that in the primary embodiment illustrated herein, the pause between sentences varies based on the sentence's word length. However, a fixed value for a pause produces acceptable results and is therefore used in alternative embodiments.
b and 6c show the details of step 606 of
In the illustrated embodiment, a phrase can either be a single word or it can consist of two words in combination, e.g., a verb and a noun. The aggregation of words to meaningful phrases is accomplished by comparing a text with a database of phrases.
Prior to step 606, step 605 (referenced in
Meaningful combinations of words can be determined by n-gram analysis. Such analysis applies statistical probability of two characters or words appearing side by side in a text. In the embodiment illustrated herein, a phrase database is used to lookup the frequency of word combinations appearing in a phrase and this frequency is used in processing logic for determining whether to combine adjacent words as a phrase in a single display element. The illustrated embodiment parses text into single-word phrases and two-word phrases. However, if a phrase consisting of two words in Chinese text would exceed the length of five characters, then the words are not displayed as part of the same display element, but rather are displayed serially in consecutive display elements. This avoids exceeding a visual span of readers and triggering a saccade.
Turning now to
Step 661 determines whether there are at least three words remaining in the sentence by determining whether wsel<the total words in the sentence —2. If the result of step 661 is no (i.e., there are fewer than three words remaining to be processed in the sentence) then the process continues with step 684 in
If the result of step 661 is yes (i.e., there are still three or more words remaining to be processed in the sentence), then steps 662-666 set variable values using the next three words to be processed as follows: Step 662 sets the variable “first_word” equal to the word corresponding to the current values of wsel (e.g., the first time step 662 executes for a sentence, the value of “first_word” would be the first_word in the sentence). Step 663 sets the variable “second_word” equal to the word corresponding to wsel+1 (i.e., the word following the word set equal to “first_word in step 663). Step 664 sets the variable “third_word” equal to the word corresponding to wsel+2 (i.e., the word following the word set equal to “second_word” in step 663). Step 665 sets the variable “current_phrase” equal to first_word followed by second_word, i.e., first_word second_word. Similarly, step 666 set the variable “next_phrase” equal to second_word followed by third_word.
Step 667 determines whether the value of current_phrase is longer than five characters. If the result of step 667 is no, then step 668 determines whether the value of current_phrase is listed in the phrase database (i.e., a database of known Chinese phrases). If yes, then step 669 determines whether the value of next_phrase is longer than five characters. If the result of step 669 is yes, then step 680 sets the next empty row in the array Phrase [ ] equal to current_phrase and step 681 sets the value of the next empty row (i.e., following the row filled in step 680) equal to third_word. If the result of step 669 is no, then step 670 determines whether the value of next_phrase is listed in the phrases database. If no, then steps 680 and 681 set the value of the next two empty rows of the array Phrase [ ] as describe above. If the result of step 670 is yes, then step 671 determines whether the value of next_phrase is a phrase that is used more frequently than the phrase that is the value of current_phrase. This example assumes that the phrase database list phrases and also lists a value for each listed phrase's use-frequency so that the frequency of use of two phrases can be compared. In a preferred embodiment, the phrase database treats idioms as phrases so that such phrases can be displayed together if the length does not exceed five characters. One example of data for idiom frequency is provided in the book “500 Common Chinese Idioms: An Annotated Frequency Dictionary,” by Jiao, Kubler, and Zhang, published by Routledge, 2011. Much larger compilations of Chinese phrases with frequency data are also available. See, for example, http://chengyu.itlearner.com/. If the result of step 671 is no, then steps 680 and 681 set the value of the next two empty rows of the array Phrase [ ] as describe above. If the result of step 671 is yes, then step 682 sets the next empty row in the array Phrase [ ] equal to first_word and step 681 sets the value of the next empty row (i.e., following the row filled in step 680) equal to next_phrase.
If the result of step 667 is yes, then step 672 sets the next empty row in the array “Phrase” equal to first_word. Step 673 then determines whether the value of next_phrase is longer than five characters. If the result of step 673 is no, then step 677 determines whether the value of next_phrase is listed in the phrase database. If yes, then step 678 sets the next empty row in the array Phrase [ ] equal to the value of next_phrase. If the result of step 673 is yes (next_phrase is longer than five characters), then step 674 sets the next empty row in the array Phrase [ ] equal to second_word. Step 675 then increments wsel by two and the process retunes to step 661. Note that in this situation, the value of first_word on the next loop will be the same as the value of third_word from the prior loop. Similarly, if the result of step 677 is no (i.e. next_phrase is not listed in the phrase database), then step 674 and 675 also proceed as just described.
If the result of step 668 is no (i.e., current_phrase is not listed in the phrase database), then step 676 sets the next row of the array Phrase [ ] equal to first_word. Step 677 then determines whether next_phrase is listed in the phrase database and processing continues from step 677 as described above.
If processing has followed paths in which steps 681, 683, or 678 are executed, then step 679 (rather than step 675) is executed. Step 679 increments wsel by three and processing returns to step 661.
If the result of step 661 is no, this implies that there are only one or two words remaining in the sentence. Processing then proceeds to step 684 in
Turning to
If the result of step 685 is no, then step 687 stores the word corresponding to wsel+1 as the value of the variable second_word. Step 688 then stores the value of the variable current_phrase as the value of first_word followed by the value of second_word (i.e., current_phrase=first_word second_word). Step 689 then determines whether current_phrase is longer than five characters. If yes, then step 690 sets the next empty row of the array Phrase [ ] equal to the value of first_word and step 691 sets the next empty row (i.e., the row following the row set in step 690) equal to the value of second_word. Processing then ends at 694.
If the result of step 689 is no, then step 692 determines whether the value of current_phrase is in the phrase database. If no, then step 690 and 691 proceed as just described above. If the result of step 692 is yes, then step 693 sets the next empty row of the array Phrase [ ] equal to current_phrase. Processing then ends at 694.
Step 702 selects and loads the next phrase-multiplier pair of values to be processed from the array Phrases [ ] (stored in step 640 shown in
Steps 704 then sets the ORP character (“orpc”) based on the value of pl by progressively testing the length until the appropriate value of orpc is selected for that phrase. The resulting orpc determination accords with the recommendations in TABLE I. Step 705 determines if pl<3. If the result of step 705 is yes, then step 706 sets the orpc to 1 (i.e., the orpc is set be the first character) and processing 700 proceeds to step 708. If the result of step 705 is no, then step 707 sets the orpc to 2 (i.e., the orpc is set be the second character) and processing 700 proceeds to step 708.
Step 708, displays ORP character (as selected by step 706 or 707) at fixed display location 201 in ORP display 200 (as illustrated by way of example in window 718). Step 709 then displays the characters to the left of the ORP character one-by-one with the offset defined by the font width (as illustrated by way of example in window 719). Step 710 then displays characters to the right of the ORP character are then displayed in the same manner (as illustrated by way of example in the window 720).
Step 711 then tests to see if the end of the text has been reached. If the result of step 711 is no, then step 712 calculates the average phrase relative duration (“apdr”) by dividing the rtd by the pit. Step 713 then calculates the default update time (“dut”) by dividing the update time (“ut”) (which is retrieved from display settings module 308 of
In most digital representations of Chinese texts, the characters all have identical width. The same width is used for displaying punctuation marks even though punctuation marks are typically not as wide as a character. Therefore, the pixel width allocated to a punctuation mark is typically greater than the visible width of the display punctuation mark. Also, punctuation marks are typically displayed closer to the left edge of the allocated pixel width than to the right edge. The visible width of a word with a punctuation mark in the end can be shorter than the sum of the word's character spaces. This deviation should be considered when determining the ORP based on proportionate position relative to the width of the word in pixels.
If no punctuation marks are shown together with the word or phrase in the display element, then a formula for calculating the ORP Offset can be written as follows:
ORP Offset=(Width*0.3)
“Width” is the total width (in pixels) of the phrase in the display element.
If punctuation marks are shown together with the phrase, then the ORP Offset can be calculated with the following formula:
ORP Offset=(Width+Right Punctuation Mark Width—Left Punctuation Mark Width)*0.3
“Right Punctuation Mark Width” and “Left Punctuation Mark Width” describe the width in pixels for the punctuation marks. The width of the displayed portion of a single punctuation mark is generally less than a character's displayed width. Therefore, in calculating the displayed width of a punctuation mark, it might be necessary to trim, i.e. reduce, the value of the display width amount that is normally allocated to a full character. In some cases, two or more punctuation marks might be displayed consecutively after and/or before the phrase. In one embodiment, the relevant variables, i.e., Right Punctuation Mark Width and/or Left Punctuation Mark Width are determined by calculating a combined displayed width of adjacent punctuation marks (on the left and/or right end of the displayed phrase). This can be done, in one embodiment, as follows: adding the full reserved width for any inner punctuation marks (i.e., a mark adjacent to the outermost mark) to the reserved width for the outermost punctuation mark (e.g., the further to the left or furthest to the right), and then trimming, i.e., reducing, the value of the reserved width of the outermost punctuation mark only.
In an alternative embodiment, punctuation marks are ignored for purposes of determining the ORP offset. In such an embodiment, punctuation marks can be shown with the phrase; however, positioning of the phrase is determined using only the above Width value, which provides the displayed width of the phrase (including only the characters in the phrase without punctuation marks). The ORP Offset is then used to determine where to place the beginning pixel of the phrase's first Chinese character (rather than using the ORP Offset to determine where to place the beginning pixel of any left-side punctuation mark). Any left-side punctuation marks would be displayed to the left of that character and any right-side punctuation marks would be displayed to the right of the phrase's last character.
Step 801 receives text from the ORP text processor 303 or from text data store 311. Step 802 selects and loads the next phrase-multiplier pair of values to be processed from the array Phrases [ ] (stored in step 640 shown in
Step 804 determines if the current phrase includes punctuation marks. If the answer in step 804 is no, then step 805 determines the first pixel position for the display of the phrase in RSVP display 200 (an exemplary ORP display 200 is shown just below step 805 in
When the beginning pixel is displayed at the ORPRO from fixed display location 201, the ORP of the phrase, which in this case is identified as the optimal proportionate position along the width of the phrase (rather than being identified as a particular character), will be displayed at fixed display location 201.
Step 812 tests to see if the end of the text has been reached. If the result of step 812 is no, then step 813 calculates the average phrase relative duration (“apdr”) by dividing the rtd by the pit. Step 814 then calculates the default update time (“dut”) by dividing the update time (“ut”), which is retrieved from the display settings module 308, by the apdr. Step 815 then sets a timer and the phrase is displayed for a time equal to the product of the relative time multiplier and the dut. Step 816 directs that the next phrase-multiplier pair is selected and processing 800 returns to step 802 so that the next phrase-multiplier can be processed. If the result of step 812 is yes, then display processing 800 ends at step 817.
The example of
As an alternative to processing all text before beginning the display processing, it is possible to display each display element (e.g., a word) after it has been processed. Referencing such an alternative in the context of modifying the processing order of steps shown in
Referencing such an alternative in the context of modifying the processing order of steps shown in
In serial text display, saccades are most noticeably a problem for words that are five or more characters in length. Therefore, a preferred embodiment of the present invention places the ORP at the fixed display location at least for display elements that are five or more characters in length. For such display elements (and for display elements of length four), the optimal recognition position is off-center toward the beginning of the display element (from the perspective of a reader of the display). However, while display of the ORP at the fixed display location for words that are fewer than five characters length is not necessary for achieving the benefits of particular embodiments of the invention, in a preferred embodiment, words of lengths four or less are also displayed using the ORP character position method or the ratio position offset method (which, for word lengths of three or less, will not necessarily result in the fixed display location being off center toward the beginning of the word). However, when such words (length four or less) are displayed at the fixed display location, exactly which character or proportionate position of these words is displayed at the fixed display location is not necessarily critical for minimizing saccades. At the same time, however, displaying words of length four or less such that their optimal recognition position is at the fixed display location does provide some smoothing benefit in that eye displacement from one word to the next is reduced, and therefore a preferred embodiment places an optimal recognition potions of all words, including words of length four or less at the fixed display location.
Other embodiments would enable the invention to be deployed on a variety of electronic devices, such as various computer operating systems, mobile phone operating systems, video gaming platforms, and portable electronic devices such as digital watches, cameras, and music players. While the basic embodiment described herein provides for ORP processing of text that is stored locally on the electronic device, it is possible that the text in a standard format or preprocessed ORP format could be streamed to the electronic device from a server via a wired or wireless network connection. While a preferred embodiment of the present invention is the display of text on small displays, if a large display is available then it would be possible to display the full text in a companion display, such that the progress through the text can be indicated by a moving highlight that corresponds to the word being currently display in the ORP Display.
Method(s), computer program product(s), and/or system(s) for displaying text comprising Chinese characters by at least:
The embodiment of paragraph [00107] wherein one or more of the factors further includes use-frequency of the display element phrase.
The embodiment of paragraph [00107] wherein one of more of the factors further includes use-frequency of one or more words in the phrase.
The embodiment of paragraph [00107], [00108], or [00109] wherein parsing comprises parsing the text into words and then parsing the text into phrases comprising one or two words of the text.
The embodiment of paragraph [00107], [00108], or [00109] wherein parsing comprises parsing the text into words and then parsing the text into phrases by, for a set of three consecutive words in the text, making a first determination regarding whether a first and second word comprise a first phrase that is five characters or less in length, making a second determination regarding whether a second and third word comprise a second phrase that is five characters or less, and, if the first determination and the second determinations are yes, then making a third determination regarding whether the first phrase is used more frequently than a second phrase.
The embodiment of paragraph [00111] wherein if the first phrase is used more frequently than the second phrase, then parsing the three words into one phrase comprising the first and second word and another phrase comprising the third word and if the second phrase is used more frequently than the first phrase, then parsing the three words into one phrase comprising the first phrase and another phrase comprising the second and third word.
The embodiment of paragraph [00107], [00108], or [00109] wherein the embodiment displays one or more visual indicators marking a fixed location on the electronic display and the embodiment displays at least some of the phrases such that an optimal recognition position of the phrase is displayed at the fixed display location.
The embodiment of paragraph [00113] wherein, for a phrase that is four or five characters in length, the phrase is displayed with the second character at the fixed display location.
The embodiment of paragraph [00107], [00108], or [00109] wherein the embodiment displays one or more visual indicators marking a fixed location on the electronic display and the embodiment displays at least some of the phrases such that a location that is between 0.2 and 0.4 of the display element's visual width is placed at the fixed display location.
The embodiment of paragraph [00107], [00108], or [00109] wherein the embodiment displays one or more visual indicators marking a fixed location on the electronic display and the embodiment displays at least some of the phrases such that a location that is off-center towards the beginning of the display element's visual width is placed at the fixed display location.
While the present invention has been particularly described with respect to the illustrated embodiments, it will be appreciated that various alterations, modifications and adaptations may be made based on the present disclosure and are intended to be within the scope of the present invention. While the invention has been described in connection with what are presently considered to be the most practical and preferred embodiments, it is to be understood that the present invention is not limited to the disclosed embodiments but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the scope of the underlying principles of the invention as described by the following claims.
This application claims the benefit of U.S. Provisional Application Ser. No. 62/013,441 file Jun. 17, 2014. The entire contents of that application is hereby incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 62013441 | Jun 2014 | US |
