Patent Application
20060066567
The present invention generally relates to display systems and, more particularly, relates to controlling the display of information to a viewer, such as a driver of a vehicle to minimize driver distraction.
Automotive vehicles are increasingly being equipped with various electronic entertainment and infotainment systems generally referred to herein as mobile multimedia (MMM) devices. For example, automotive personal computing (PC) devices have been installed on vehicles to allow personal computing, web browsing, and Internet access. Radio and satellite communication devices, such as radio data systems (RDS) have also been installed on vehicles.
Various vehicle installed mobile multimedia (MMM) devices also allow for the communication of information to occupants in the vehicle. Many mobile multimedia devices typically include a human machine interface (HMI) for enabling a user to interface with the device. For example, the HMI typically includes a display for viewing messages, navigational maps, and other information. As the complexity of mobile multimedia devices generally increases, displays are generally made available with more information that is to be presented in a relatively small conformal display space. In order to reduce cost and utilize space efficiently, vehicle manufacturers often prefer smaller displays. However, in order for small displays to provide large amounts of text information, the text may be presented on the display as scrolling text. For example, in an RDS radio system, the RDS radio information, which may include artist and song title, may be presented as text that is scrolled across the display. Scrolling text is text that is scrolled vertically or horizontally on the display as a single or plurality of lines of text. Thus, only a portion of the text message may be viewable at a given time.
While scrolling text offers the ability to present large amounts of information in a smaller display space, the scrolling text can annoy the driver and pose a distraction, since the scrolling text is generally displayed at a predetermined rate. When scrolling text is presented on a display, the driver may feel a desire to keep his or her eyes fixed on the display to acquire the text message information. Excessive driver attention to scrolling text may therefore cause a distraction. On the other hand, the driver may be unable to acquire important information if the driver focuses attention away from the display. This may lead to frustration by the driver.
It is therefore desirable to provide for a system and method for presenting scrolling text information on a display in a manner that minimizes distraction and effectively presents information to a viewer. It is further desirable to provide for a system and method for presenting scrolling text information on a vehicle display in a manner that is convenient to the driver of a vehicle.
The present invention provides for a display system and method of controlling the display of scrolling text information on a display in a manner that generally minimizes driver distraction. The display system includes a display for displaying information comprising scrolling text. The system also includes a monitor including an imaging camera oriented to capture images of a person including at least a portion of the head of the person and determining an orientation vector. The system further includes a controller for determining whether the person is expected to be viewing the display based on the orientation vector. The controller further controls the presentation of scrolling text on the display based on whether the person is expected to be viewing the display.
According to a further aspect of the present invention, the display system is particularly well-suited for use on a vehicle to present scrolling text to the driver of the vehicle, while minimizing distraction to the driver. When the driver is not expected to be viewing the display, the scrolling text display is paused so that new text is not presented until the driver redirects attention to the display.
According to another aspect of the present invention, a method is provided for controlling scrolling text on a display based on expected viewing by a person, such as a driver of a vehicle. The method includes the steps of presenting scrolling text on a display, and capturing images of a person including at least a portion of the head of the person. The method also includes the steps of determining an orientation vector based on the captured images, and determining whether the person is expected to be viewing the display based on the orientation vector. The method further includes the step of controlling the presentation of scrolling text on the display based on whether or not the person is expected to be viewing the display.
These and other features, advantages and objects of the present invention will be further understood and appreciated by those skilled in the art by reference to the following specification, claims and appended drawings.
The present invention will now be described, by way of example, with reference to the accompanying drawings, in which:
Referring now to
Centrally mounted within the dash 12 is an electronic device 20, such as a mobile multimedia device which is generally accessible to both the driver of the vehicle and another front passenger. The electronic device 20 may include any of a variety of information and entertainment devices commonly known in the art. For example, the electronic device may include any one of or a combination of the following devices: an automotive personal computing device, a web browser, an Internet access device, a satellite communication device, a mobile multimedia device, a radio, a television, a DVD player, a navigation system, a game system, a phone/address book lookup system, and any other types of electronic devices which employ a display for displaying text.
The infotainment device includes human machine interface (HMI) inputs 26 for allowing occupants of the vehicle to interface via input selections. The electronic device 20 also includes an HMI output in the form of a display 22 for displaying messages and other information. The display 20 may include any display that is configurable to display scrolling text 24, such as a liquid crystal display (LCD), light emitting diode (LED) display, plasma display, and tube displays. The scrolling text may be scrolled vertically or horizontally as single or multiple lines of text. With scrolling text, only a portion of the text message may be viewable at a given time. This enables a large amount of text information to be displayed on the display 22 than would otherwise be viewable in a single display frame.
The display system also has a monitor (gaze monitor) including one or more video imaging cameras for generating video images of the driver of the vehicle 10. The acquired video images are processed for tracking the head portion of the driver to monitor facial characteristics, such as the driver's eyes, to determine a driver's orientation vector (e.g., gaze vector) based on eye gaze and/or head orientation. The driver gaze vector is used to determine whether the driver is expected to be viewing the display 22. If the driver is expected to be viewing the display 22, the scrolling text 24 may be presented on the display 22 at a first rate. If the driver is not expected to be viewing the display 22, the scrolling text 24 is presented at a second rate, which may be paused (i.e., rate equals zero), so that the driver can direct attention elsewhere and not miss the displayed message.
As seen in
In
Referring to
While the imaging camera 30 is shown mounted in the dash 12 in a position between the display 22 and a point directly in front of the driver's seat 28 of the vehicle, it should be appreciated that the imaging camera 30 may be located elsewhere. For example, the imaging camera 30 may be located in or on top of the dash 12 in a region generally in front of the driver's seat 28, or may be mounted interior on or near the rearview mirror, or may be mounted in the A-pillar, or may be located elsewhere in position suitable to capture images of the head 34 of the driver 32 sufficient to determine an expected driver gaze {overscore (g)}.
The video camera 30 may include a CCD/CMOS active-pixel digital image sensor. One example of a CMOS active-pixel digital image sensor is Model No. PP-0330, commercially available from Photobit, which has a resolution of about 640 H×480V. It should be appreciated that other cameras, including less costly and less sophisticated video cameras, may be employed.
Referring to
The gaze monitor 40 also has a camera control function via RS-232 logic 44 which allows for control of the video imaging camera 30. Control of the video imaging camera 30 may include automatic adjustment of the pointing orientation of the video imaging camera 30. For example, the video imaging camera 30 may be repositioned to focus on an identifiable feature, and may scan a region in search of an identifiable feature, including the driver's face and, more particularly, one or both eyes. The gaze vector output is communicated via a serial output 49 to an HMI controller 50. Further control of the camera 30 may include adjustment of focus and magnification as may be necessary to track one or more identifiable features.
The HMI controller 50 includes a microprocessor-based controller having a microprocessor 52 and memory 54. Memory 54 may include RAM, ROM, EEPROM, and other memory as should be evident to those skilled in the art. The HMI controller 50 is programmed to include a scrolling text control routine 80 for controlling the presentation of scrolling text on display 22 based on the determined driver gaze vector {overscore (g)}.
The HMI controller 50 provides output control signals to display 22 to control the scrolling text to allow scrolling text to be displayed when the driver is determined to be viewing the display 22 and to pause (stop) the scrolling text displayed when the driver is not determined to be viewing the display 22. This enables the driver to direct attention elsewhere such that the displayed information is frozen and is not missed. The driver may subsequently redirect attention back to the display and, when the driver is determined to be viewing the display, the scrolling text is again presented on the display 22.
In the embodiment shown, the video images are processed by a gaze monitor 40 and are further processed by a separate HMI controller 50 to control scrolling text displayed on the display 22. The use of a separate gaze monitor 40 and HMI controller 50 enables the present invention to be employed on vehicles equipped with some portion of the control devices, such as a vehicle already equipped with a gaze monitor. While separate control devices 40 and 50 are shown and described herein, it should be appreciated that the video image processing and scrolling text control may be handled by a single controller, or may be provided in any of a number of control devices having suitable memory and processing capability.
In
Given the tracked facial features, routine 60 calculates the face orientation vector of the driver's head {overscore (h)}=(hx, hy, hz) with regard to the vehicle, and further calculates the gaze vector with regard to the driver's face orientation in step 68. Finally, in step 70, routine 60 uses the face orientation vector {overscore (h)} and gaze vector with regard thereto to determine the eye gaze vector {overscore (g)}=(gx, gy, gz) with regard to the vehicle (car), before returning to step 64. Accordingly, routine 60 determines an expected eye gaze vector {overscore (g)} of the driver of the vehicle based on the face orientation and gaze vector.
Referring to
Once the gaze vector {overscore (g)} has been computed, routine 80 proceeds to decision step 86 to determine if the gaze vector {overscore (g)} falls within a text display region. The text display region may be a predetermined region in the general area of the display that presents the scrolling text. The text display region may be configured to be the near identical viewing region of the display or may include an enlarged area around the display.
If the gaze vector falls within the text display region, routine 80 enables the displayed text to be scrolled in step 90. This allows for the continuous presentation of scrolling text on the display. If the gaze vector {overscore (g)} does not fall within the text display region, indicative of the driver not expected to be viewing the display, then routine 80 pauses the text scrolling in step 88. This essentially freezes the text on the display to enable the driver to direct attention elsewhere and, upon the driver returning to view the display, the same text information remains on the display, at least momentarily. It should be appreciated that routine 80 may include a delay before re-enabling the text to scroll, such that the driver may re-read the line of text paused on the display before the scrolling of the text occurs.
Accordingly, the display system of the present invention advantageously controls the presentation of scrolling text on a display in a manner that minimizes distraction and effectively presents the information to a viewer. The display system is particularly useful for presenting scrolling text information on a vehicle display in a manner that is convenient to the driver of the vehicle and makes the use of the display less of a distraction. While the display system is shown and described herein in connection with controlling scrolling text based on viewing of a driver of a vehicle, it should be appreciated that the display system may likewise be applicable to controlling any of a number of displays based on viewing by any person on or off a vehicle.
It will be understood by those who practice the invention and those skilled in the art, that various modifications and improvements may be made to the invention without departing from the spirit of the disclosed concept. The scope of protection afforded is to be determined by the claims and by the breadth of interpretation allowed by law.
