Patent Application
20070040810
This invention relates to display devices, in particular to methods and user input systems for use in display devices.
Display devices, including but not limited to, digital still cameras, video cameras, cellular telephones and the like conventionally use displays in a fixed position within a device body. Alternatively, it is known to provide displays that are fixed within a housing of a type that is joined to but movable relative to a body of a display device such as is done with some types of video cameras. A user of such a display device controls the device by way of external user input controls such as buttons, joysticks, dials, wheels, jog dials and the like. Such user input controls are placed around the periphery of the display or on other surfaces of the display device, such as on front, top, bottom, back or sides. These controls occupy a certain amount of surface area on the display device thus, the overall size of a display device is in part determined by the size of the display and by the number of independent external controls used to operate the display device.
For example,
For selecting certain previously captured pictures for review, menu button 24, navigation button 30, and select button 26 can be used to select a review function from the menu. When the review function has been selected, navigation through the pictures is accomplished by pressing navigation button 30 to the right or left towards arrows 17 and/or 19 respectively.
As the technology used in display devices becomes more capable and as displays become less expensive, there is a desire to offer display devices with larger displays. There is also a concomitant desire to provide display devices that offer a greater range of features which in turn demands a greater variety and/or number of controls. As a result of these influences, many display devices are becoming proportionately larger. However, there is also a desire for such devices to become smaller and lighter so as to provide portability and convenience advantages. These competing desires have caused display devices to be developed that devote more of the external surface area of a display device for the display and that therefore have a smaller proportion of external surface area of the display device available for use in locating the controls. Accordingly, fewer controls are being incorporated in display devices with the controls being used for multiple, often unrelated, purposes such as where different controls are used for different purposes in different modes of operation. This however, is confusing to many users.
Another solution to this problem is to use a special type of display having a touch screen. A touch screen display has special transparent surface that can sense when a finger or stylus contacts the surface and can provide control signals that can be used to control device functions. Several types of touch screens are available such as resistive touch screens having a matrix of resistors that change resistance when touched, and capacitive touch screen having a matrix of capacitors that change capacitance when touched.
On/off button 22 is present to activate the prior art digital camera 12 of
For reviewing pictures already taken with the prior art digital camera 12 of
Thus, touch screen displays 36 save space on a display device by reducing the number of external display controls thereby allowing a touch screen display 36 to occupy a greater proportion of the exterior surface of a display device. However, there are some disadvantages for using touch screen display 36 in a display device. For example, the cost of touch screen display 36 is comparatively high for many display devices and such touch screens are often vulnerable to damage from incidental contact causing such a display to wear and fail well before the useful life of the digital camera 12 or other display device in which the display is mounted has expired. Further, repeated finger contact with the touch screen can leave an unattractive pattern of fingerprints on the display which can be difficult to clean without risking damage to the touch screen display 36. Finally, many such screens are particularly vulnerable to damage electronic discharge and other environmental contaminates.
Accordingly, what is desired is a way to use the portion of an external surface of a display device to sense user input actions and to generate signals in response thereto for control of the display device so that the number of controls external to the display can be minimized while still providing a convenient user input scheme with a robust interface in a low cost design.
In one aspect of the invention, a display device is provided. The display device comprises a body having an opening to a display receiving area; a display joined to the body within the display receiving area; and a generally transparent contact element positioned between the opening and the display so that at least a part of an image presented by the display is viewed through the contact element. At least two force sensitive elements are between the contact element and the display receiving area, and each force sensitive element is adapted to generate a signal when a force has been applied to the contact element. A controller receives the signals and determines a user input action based upon the signals received. The force sensitive elements are adapted to detect the application of force along different axes and to generate signals that the controller can use to determine when a force has been applied to the contact element and along which of the different axes the force has been applied.
In another aspect of the invention, a display device comprises a body having a display area with a display therein, a generally transparent contact element joined to the body for movement between a neutral position and two separate force applied positions into which the contact element can be moved within the display receiving area when a force is applied and arrayed so that images presented by the display to viewed therethrough. A plurality of force sensitive elements is between the contact element and the display receiving area. Each force sensitive element is adapted to sense movement of the contact element into either of the force applied positions; and a controller to determine a user input action based upon the force applied to the force sensitive elements by the contact element. Wherein movement of the contact element into one of two separate force applied positions require movement of the contact element along a different axis than movement of the display into the other one of two force applied positions.
In yet another aspect of the invention, a display device comprises a body having a display receiving area; a display joined to the body within the display receiving area; a plurality of force sensing elements positioned in the display receiving area in association with the display so as to sense the application of force to the display along at least two separated axes; and a controller to determine a user input action based upon sensed application of force of the display.
In still another aspect of the invention, a method is provided for operating a display device having a contact element positioned within a display receiving area on a body. In accordance with the method, the application of force by the contact element against structures holding the contact element to the display receiving area at least along two different possible axes of movement and determining a user input action based upon a sensed application of force to the contact element.
Light from the scene that is focused by lens system 123 onto image sensor 124 is converted into image signals representing an image of the scene. Image sensor 124 can comprise a charge couple device (CCD), a complimentary metal oxide semiconductor (CMOS) sensor, or any other electronic image sensor known to those of ordinary skill in the art. The image signals can be in digital or analog form.
Signal processor 126 receives the image signals from image sensor 124 and transforms each image signal into a digital image in the form of digital data. In the embodiment illustrated, signal processor 126 has an analog to digital conversion capability. Alternatively, a separate analog to digital converter (not shown) can be positioned between image sensor 124 and signal processor 126 to convert image signals into a digital form. In this latter embodiment, signal processor 126 can comprise a digital signal processor adapted to convert the digital data from such an analog to digital converter into a digital image. The digital image can comprise one or more still images, multiple still images and/or a stream of apparently moving images such as a video segment. Where the digital image data comprises a stream of apparently moving images, the digital image data can comprise image data stored in an interleaved or interlaced image form, a sequence of still images, and/or other forms known to those of skill in the art of digital video. Signal processor 126 can apply various image processing algorithms to the image signals when forming a digital image. These can include, but are not limited to, color and exposure balancing, interpolation and compression.
A controller 132 controls the operation of display device 100, including, but not limited to, image capture system 122, display 129 and a memory 140 during imaging operations. Controller 132 causes image sensor 124, optional lens driver 125, signal processor 126, display 129 and memory 140 to capture, process, store and/or display images in response to signals received from a user input system 134, data from signal processor 126 and data received from optional sensors 136 and/or signals received from a communication module 149. Controller 132 can comprise a microprocessor such as a programmable general-purpose microprocessor, a dedicated microprocessor or micro-controller, an arrangement of discrete elements, or any other system that can be used to control operation of display device 100.
Controller 132 cooperates with user input system 134 to allow display device 100 to interact with a user. User input system 134 can comprise any form of transducer or other device capable of receiving an input from a user and converting this input into a form that can be used by controller 132 in operating display device 100. For example, user input system 134 can comprise controls such as a touch screen input, a touch pad input, a 4-way switch, a 6-way switch, an 8-way switch, a stylus system, a trackball system, a joystick system, a voice recognition system, a gesture recognition system or other such systems.
In the embodiment shown in
Sensors 136 are optional and can include light sensors, position sensors and other sensors known in the art that can be used to detect conditions in the environment surrounding display device 100 and to convert this information into a form that can be used by controller 132 in governing operation of display device 100. Sensors 136 can include, for example, a range finder of the type that can be used to detect conditions in a scene such as distance to subject. Sensors 136 can also include biometric sensors adapted to detect characteristics of a user for security and affective imaging purposes.
Controller 132 causes an image signal and corresponding digital image to be formed when a trigger condition is detected. Typically, the trigger condition occurs when a user depresses capture button 142 however, controller 132 can determine that a trigger condition exists at a particular time, or at a particular time after capture button 142 is depressed. Alternatively, controller 132 can determine that a trigger condition exists when optional sensors 136 detect certain environmental conditions such as a pulse of infra red light.
Controller 132 can also be used to generate metadata in association with each image. Metadata is data that is related to a digital image or a portion of a digital image but that is not necessarily observable in the image data itself. In this regard, controller 132 can receive signals from signal processor 126, camera user input system 134, and other sensors 136 and, optionally, generates metadata based upon such signals. The metadata can include but is not limited to information such as the time, date and location that the image was captured, the type of image sensor 124, mode setting information, integration time information, taking lens unit setting information that characterizes the process used to capture the archival image and processes, methods and algorithms used by display device 100 to form the archival image. The metadata can also include but is not limited to any other information determined by controller 132 or stored in any memory in display device 100 such as information that identifies display device 100, and/or instructions for rendering or otherwise processing the digital image with which the metadata is associated. The metadata can also comprise an instruction to incorporate a particular message into a digital image when presented. Such a message can be a text message to be rendered when the digital image is presented or rendered. The metadata can also include audio signals. The metadata can further include digital image data. The metadata can also include any other information entered into display device 100. Controller 132 will also typically be adapted to use, process, edit and store metadata that is provided with images that are not captured by display device 100.
Digital images and optional metadata can be stored in a compressed form. For example, where the digital image comprises a sequence of still images, the still images can be stored in a compressed form such as by using the JPEG (Joint Photographic Experts Group) ISO 10918-1 (ITU-T.81) standard. This JPEG compressed image data is stored using the so-called “Exif” image format defined in the Exchangeable Image File Format version 2.2 published by the Japan Electronics and Information Technology Industries Association JEITA CP-3451. Similarly, other compression systems such as the MPEG-4 (Motion Pictures Export Group) or Apple Quicktime™ standard can be used to store digital images that are in a video form. Other image compression and storage forms can be used.
The digital images and metadata can be stored in a memory such as memory 140. Memory 140 can include conventional memory devices including solid state, magnetic, optical or other data storage devices. Memory 140 can be fixed within display device 100 or it can be removable. The digital images and metadata can also be stored in a remote memory system 147 that is external to display device 100 such as a personal computer, computer network or other imaging system.
In the embodiment shown in
Signal processor 126 optionally also uses images signals or the digital images to form evaluation images which have an appearance that corresponds to captured image data and are adapted for presentation on display 129. This allows users of display device 100 to observe digital images that are available in display device 100. For example, images that have been captured by image capture system 122, that are otherwise stored in a memory, such as memory 140, or that are received by way of communication module 149. Display 129 can comprise, for example, a color liquid crystal display (LCD), organic light emitting display (OLED) also known as an organic electroluminescent display (OELD) or other type of video display.
Signal processor 126 and controller 132 also cooperate to generate other images such as text, graphics, icons and other information for presentation on display 129 that can allow interactive communication between controller 132 and a user of display device 100, with display 129 providing information to the user of display device 100 and the user of display device 100 using user input system 134 to interactively provide information to display device 100. Display device 100 can also have other displays such as a segmented LCD or LED display (not shown) which can also permit signal processor 126 and/or controller 132 to provide information to a user. This capability is used for a variety of purposes such as establishing modes of operation, entering control settings, user preferences, and providing warnings and instructions to a user of display device 100. Other systems such as known systems and actuators for generating audio signals, vibrations, haptic feedback and other forms of signals can also be incorporated into display device 100 for use in providing information, feedback and warnings to the user of display device 100.
Typically, display 129 has less imaging resolution than image sensor 124. Accordingly, signal processor 126 reduces the resolution of image signal or digital image when forming evaluation images adapted for presentation on display 129. Down sampling and other conventional techniques for reducing the overall imaging resolution can be used. For example, resampling techniques such as are described in commonly assigned U.S. Pat. No. 5,164,831 “Electronic Still Camera Providing Multi-Format Storage Of Full And Reduced Resolution Images” filed by Kuchta et al., on Mar. 15, 1990, can be used. The evaluation images can optionally be stored in a memory such as memory 140. The evaluation images can be adapted to be provided to an optional display driver 128 that can be used to drive display 129. Alternatively, the evaluation images can be converted into signals that can be transmitted by signal processor 126 in a form that directly causes display 129 to present the evaluation images. Where this is done, display driver 128 can be omitted.
Display device 100 captures digital images using image sensor 124 and other components of image capture system described above. Imaging operations that can be used to capture digital images include a capture process and can optionally also include a composition process and a verification process.
During the optional composition process, controller 132 causes signal processor 126 to cooperate with image sensor 124 to capture digital images and present a corresponding evaluation images on display 129. In the embodiment shown in
The capture process is executed in response to controller 132 determining that a trigger condition exists. In the embodiment of
During the verification process, the corresponding evaluation image is supplied to display 129 and is presented for a period of time. This permits a user to verify that the digital image has a preferred appearance.
Digital images can also be received by display device 100 in ways other than image capture. For example, digital images can by conveyed to display device 100 when such images are recorded on a removable memory. Alternatively digital images can be received by way of communication module 149. For example, where communication module 149 is adapted to communicate by way of a cellular telephone network, communication module 149 can be associated with a cellular telephone number or other identifying number that for example another user of the cellular telephone network such as the user of a telephone equipped with a digital camera can use to establish a communication link with display device 100 and transmit images which can be received by communication module 149. Accordingly, there are a variety of ways in which display device 100 can receive images and therefore it is not essential that display device 100 have an image capture system so long as other means such as those described above are available for importing images into display device 100.
In the embodiment of
Also shown in
Force sensitive elements 150, 152, 154 and 156 can be pushbutton switches or can comprise any structure or assembly that can sense the application force thereto and that can generate a signal or that can cause a detectable signal to be generated. A variety of exemplary embodiments force sensitive elements are discussed hereinafter, however, force sensitive elements usable with this invention are not limited to these exemplary embodiments.
When the user wishes to access a camera function other than taking a picture, the user can press on display 129 over one or more of force sensitive elements 150, 152, 154, 156. For instance, to access a main menu, the user can press display 129 in the center applying a downward force along each of axes A1, A2, A3, and A4 causing all four force sensitive elements 150-156 to be depressed at the same time. Controller 132 will recognize that the depression of all four force sensitive elements 150-156 at once is a signal that a main menu is to be displayed.
After the zoom function is selected, zoom control menu 169 shown in
For reviewing pictures already taken, a user of camera 102 can return to main menu 158 and select a review function using by pressing on another portion of display 129. The user can then navigate through the pictures by pressing the right and left sides of display 129 or by otherwise pressing particular portions of display 129.
After the desired functions have been selected, the user can return to main menu 158 by executing one or more of pre-programmed depressions of display 129. Once main menu 158 is displayed, the user could selectively press on display 129 toward force sensitive element 154, causing force sensitive element 154 to send a signal to controller 132 causing controller 132 to enter an image capture mode.
To prevent erroneous readings of depressions of force sensitive elements 152, 154, 156, and 158, controller 132 can be adapted to recognize, as a control signal, only those sensed depressions that last continuously for at least a minimum amount of time, such as for example, between 2 and 300 milliseconds. Alternatively, controller 132 can require a predetermined amount of force to be applied to each force sensitive element. Further, a time delay could be incorporated into the control program to read if only one switch had been depressed or that more than one switch had been depressed. This time delay may be, for example, only a few milliseconds or several hundred milliseconds and is determined by the designers.
In the embodiments illustrated, a resilient linkage 146 is shown as a layer of resiliently deformable material such as a sponge rubber material. Resilient linkage 146 helps a contact element 130, such as display 129, return to a level or other default orientation after force has been applied. Resilient linkage 146 can comprise a sponge rubber material that covers the entire area underneath display 129 except where force sensitive elements and fulcrum, if used, are positioned. The sponge rubber material can be adhered to display receiving area 148 and also to display 129.
Alternatively, resilient linkage 146 can be made of some type of resilient material other than sponge rubber, such as an elastomer. Other structures for attaching a contact element 130, such as display 129, to display device 100 can be used so long as resilient linkage 146 continues to offer a resilient response to pressure that is applied to display 129. For example, in one embodiment, resilient linkage 146 can be provided by a combination of a movable support such as a pivot (not shown) that allows display 129 to move within a range of position, and force sensitive elements 150, 152, 154 and 156 that are adapted to resiliently bias display 129 from positions within the range to a neutral position after an applied force moves display 129 to other positions within the range.
Returning now to
In one application of this embodiment, display 129 can be used to enter an image rotation mode and can be rotated to intuitively indicate a desire to rotate a captured image. As is illustrated in
Force sensitive elements 150, 152, 154, 156 and 182, 184, 186, 188, 190, 192, 194 can take a variety of forms. In certain embodiments, force sensitive elements 150, 152, 154, 156 and 182, 184, 186, 188, 190, 192, 194 can comprise any materials that can be resiliently expanded, compressed or otherwise shape changed in response to pressure that is applied thereto and that changes characteristics that can be detected by controller 132 when the shape is changed. For example, such as by changing capacitance, resistance, surface conductivity, or by generating a voltage or current signal.
Alternatively, force sensitive elements can be adapted to sense force with a minimum of shape change, so that a force can be applied to display 129 that causes generally insubstantial movement of display 129, but that transmits a force to the force sensitive elements that causes the force sensitive elements to generate signals that can be detected by controller 132 and used to determine the application of force. Here too, materials or structures that deflect only minor amounts in response to force, but that generate a signal that can be detected by controller 132, can be used. For example, a force sensitive element of this type can comprise a piezoelectric crystal or an arrangement of conductive plates that provide a large capacitive differential across in response to small variations in proximity such as may be generated by an application of force to parallel conductors separated by a dielectric that can be compressed by an applied force.
It will be appreciated that, in certain embodiments of the invention, it can be useful to provide a contact element 130, such as display 129, that can move within receiving area 148 wherein the extent of such movement can be sensed without necessarily maintaining contact between display 129 and the force sensing elements. Such an arrangement of force sensitive elements can be provided by mounting display 129 on a resilient linkage 146 that biases display 129 into a neutral position and resists movement of display 129 when a force is supplied thereto and by providing one or more positional sensors that are each adapted to detect when display 129 has been moved from the neutral position along at least one of two detectable axes of movement to an activation position. Such a combination is capable of detecting the application of force to display 129 in that display 129 cannot be moved without overcoming the bias force applied by resilient linkage 146. There are a variety of sensors that can be used for this purpose including optical, electrical switches or electromechanical switches. A principal advantage of this approach is that it is not necessary to provide sensors that are in and of themselves adapted to sense an application of force. Rather, in this embodiment, it is a combination of such sensors with a resilient linkage 146 that resists the application of force to enable one or more force sensitive elements that can sense an application of force display 129.
In the embodiment illustrated
As display 129 is moved against a bias supplied by a resilient member (not shown) from an initial position shown in
In the above described embodiments, contact element 130 has been shown in the form of a display 129 that a user of display device 100 can physically contact in order to provide user input. This advantageously provides the ability to provide a wide variety of virtual user input controls for display device 100 and to provide dynamic feedback to a user during user input actions or minimizing the cost of display device 100. However, there may be applications where it is not desirable to apply force to display 129 such as where there is a risk that such applied force can damage display 129 or that such applied force will cause display 129 to operate in an unpleasing manner. Accordingly,
In the embodiments of
Further, it will be appreciated that, any of the above described embodiments of pressure sensitive elements can be adapted to provide signals that are indicative of an amount of force applied to the display and in such embodiments, controller 132 can be adapted to use such signals for a variety of purposes. For example, in one aspect controller 132 can execute particular functions at a rate or to an extent determined by the amount of force applied to the display. For example, if a user of a display device 100 such as camera 102 wishes to review a set of images, the user can select the image review function for example from main menu 158 which can cause controller 132 to present one or more images on display 129. A user can scroll through the presented images by applying a force to display 129 along an axis. While the user does this, controller 132 can monitor the amount of force applied any given time and can adjust the rate at which images are scrolled through the display 129 in proportion to the amount of force applied. The rate can be linearly related to the amount of force applied for can be related to the amount of force applied by some other non-linear relation.
The invention has been described in detail with particular reference to certain preferred embodiments thereof, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.
