Patent Grant
8203537
The present invention contains subject matter related to Japanese Patent application JP 2007-233369 filed in the Japanese Patent Office on Sep. 7, 2007 and the entire contents of which being incorporated herein by reference.
(1) Field of the Invention
The present invention relates to a user interface device and a personal digital assistant (PDA) which can convey information to a user according to a tactile and visual mode.
(2) Description of the Related Art
In the past, for example, an information display/input device, in which a so-called touch panel is arranged on a display of a liquid crystal panel, displays an image of a virtual key on a display screen and makes it possible to input or select the information allocated to the virtual key, by a user touching the top of the touch panel area corresponding to the display position of the virtual key with a fingertip or a nib of a stylus pen, etc.
Document 1 (JP-2004-157933 A, FIG. 1 and FIG. 9) discloses a concavo-convex formation device which includes: plural displacement elements possessing a displacement unit arranged on a transparent substrate, and a displacement power generator operable to displace the displacement unit in a separating direction from the substrate correspondingly to a control signal; a transparent film layer provided in the vicinity of these displacement elements and operable to deform a contact portion to convex when the displacement unit receives a displacement power at the contact portion concerned; and a control means operable to output a control signal to each of the displacement elements individually. In the concavo-convex formation device, a transparent substrate and a film layer are laminated with a display device to form structure of a panel, over the surface of which a touch detection means is provided to detect contact. In Document 1, a user is informed of an appointed display position (for example, a virtual key position) to be touched at the time of a touch panel operation, by displacing to a shape of convex the transparent film layer of the concavo-convex formation device.
As mentioned above, by employing the constitution which arranges a concavo-convex formation device and a touch panel on a panel surface of a display device, and enabling formation of a concavo-convex shape on the panel surface concerned, not only the communication of information in a visual mode by the displayed content on a screen but also the communication of information in a tactile mode by formation of the concavo-convex shape becomes possible to a user.
However, in such constitution as described in Document 1, the concavo-convex shape formed in a certain position on the panel surface turns typically into the same concavo-convex shape. Namely, the concavo-convex shape which can be formed in a certain position of the panel surface depends on the shape of the displacement element arranged to the position. Consequently, it is difficult to form a concavo-convex shape at a certain time for example, and to form another different concavo-convex shape at another time. In other words, in the constitution disclosed by Document 1, the information which is conveyed by a visual mode to a user by changing the displayed content on the screen can be changed; however, the information which is conveyed by a tactile mode to a user can not be changed, since the concavo-convex shape turns always into the same shape.
On the other hand, if the information which is conveyed by a tactile mode can be changed linking to the change of the displayed content concerned (namely, if a concavo-convex shape can be changed), at the same time as the information conveyed by a visual mode is changed by changing the displayed content, a user interface which is more diverse and faithful to a user will be realized.
The present invention is made in view of the above circumstances and provides a user interface device and a personal digital assistant which enable change of the information conveyed by a tactile mode as well as change of the information conveyed by a visual mode, by employing the constitution which can convey information by a tactile and visual mode to a user.
The user interface device and personal digital assistant of an embodiment of the present invention solves the subject mentioned above by including: a transparent panel unit including plural layers, each of the layers being provided with plural transparent deformation sections operable to be deformed to a specified shape by injection of fluid or discharge of fluid; a fluid channel unit possessing structure forming a first channel and a second channel in common or separately, the first channel being a transparent fluid channel operable to supply fluid to be injected to the deformation sections and the second channel being a transparent fluid channel operable to pass fluid discharged from the deformation sections; a fluid pump unit operable to perform discharge of fluid at least to the fluid channel unit or suction of fluid at least from the fluid channel unit; a display panel unit including the transparent panel unit provided over an upper surface of the display panel unit and a screen surface operable to display at least an image; and a controller operable to control injection of the fluid to the deformation sections of the transparent panel unit or discharge of the fluid from the deformation sections of the transparent panel unit, according to an image to be displayed on the screen surface of the display panel unit.
That is, according to the present invention, in the transparent panel including plural layers each of which is provided with plural transparent deformation sections deformable into a specified shape by injection or discharge of fluid, a shape formed on the transparent panel unit can be changed into various shapes by deforming the respective deformation section of each layer.
In the present invention, providing a transparent panel in which plural transparent deformation sections deformable into a specified shape by injection or discharge of fluid are arranged in each of plural layers, it becomes possible to change the information which is conveyed in a tactile mode as well as the information which is conveyed in a visual mode, by deforming each of the deformation sections of each layer to a specified shape.
Embodiments of the present invention will be described in detail based on the following figures, wherein:
Hereinafter with reference to the accompanying drawings, the embodiments of the present invention are explained in detail.
In the following embodiments, a mobile-phone terminal is explained as an example of application of a user interface device and a personal digital assistant of the present invention. It is needless to say, however, that the contents explained here are of course a mere example, and that the present invention is not limited to the example.
[Basic Constitution]
The tactile-visual UI panel 45 possesses a transparent sheet 42, plural deformation sections 41, a pump 43, and a fluid channel 44 at least.
The transparent sheet 42 is formed by transparent elastomeric material etc. possessing elasticity. The transparent sheet 42 is provided in the front (the upper, screen surface side) of a display panel 46 of the mobile-phone terminal 40, and is arranged so that the transparent sheet 42 may cover almost the whole surface of the screen surface of the display panel 46 concerned. In the case of the mobile-phone terminal of the present embodiment, a transparent touch panel (not shown) is provided in the front side (the upper surface side) of the transparent sheet 42. The transparent touch panel possesses elasticity same as the transparent sheet 42 concerned, and is arranged so that the transparent touch panel may cover almost the whole surface of the transparent sheet 42. The transparent touch panel in the present embodiment employs a panel constituted by the existing technology in which, for example, when a user touches the surface of the panel with a fingertip, the nib of a stylus pen, etc., the contact concerned is detected and the detection signal is outputted. Therefore, the illustration and detailed explanation of the transparent touch panel are omitted here.
The fluid channel 44 is formed by a tube-like transparent member, for example, through which fluid (for example, a liquid, a gas, a gelatinous material, etc.) is movable. One end of the tube of each fluid channel 44 is connected with a fluid discharge suction orifice of the pump 43, and the other end of the tube is connected with each deformation section 41. It is desirable that the fluid channel 44 concerned is formed by material which possesses flexibility of a certain degree in the longitudinal direction and in the direction orthogonal to the longitudinal direction of the tube, but does not change at least with respect to the diameter (namely, it does not expand nor contract in the diameter direction).
The pump 43 is operable to perform the discharge and suction of fluid. A so-called diaphragm micro pump is employed as an example in the present embodiment.
Each of the plural deformation sections 41 is formed by a transparent elastomeric material and possesses a predetermined shape (a circular shape in the example of
The transparent sheet 42, each of the deformation sections 41, and the fluid channel 44 may be formed by the same member. That is, for example, the transparent sheet 42 may be formed by stacking and gluing two transparent elastomeric sheets, and when stacking and gluing these two sheets together, the portion of each of the deformation sections 41 and the portion of each of the fluid channels 44 are not glued. By the method, when the fluid is discharged from the pump 43, the fluid will pass along the fluid channel 44 formed by the unglued part of the two sheets, and will be injected into the circular section of the deformation section 41 similarly formed by the unglued part of the two sheets. As the result, the circular section concerned will expand. On the other hand, when the pump 43 sucks the fluid, the circular section will contract because the fluid discharged from the interior of the circular section of the deformation section 41 returns to the pump 43 through the fluid channel 44. In the case of this example, if the width of the unglued part which forms the fluid channel 44 (namely, it corresponds to the diameter of the fluid channel) is small enough compared with the thickness of the transparent sheet 42, the expansion and contraction of the fluid channel 44 concerned in the diameter direction by the fluid can be ignored.
In
As mentioned above, according to the basic constitution of the tactile-visual UI panel 45 in the mobile-phone terminal of the present embodiment illustrated in
In the mobile-phone terminal of the present embodiment, if an image of a virtual key is displayed on a display screen so as to correspond to the place where the deformation section 41 is arranged over the tactile-visual UI panel 45, and at the same time, if the fluid is injected to the deformation section 41 of the tactile-visual UI panel 45 to deform the deformation section 41 to a convex shape, the virtual key on the display screen will actually appear visually as a key of a convex shape to a user, and the user will be able to recognize the deformation section 41 of the convex shape as an actual key, tactilely as well as visually, by touching the deformation section 41 with a finger etc.
Since the touch panel is provided over the surface of the tactile-visual UI panel 45 in the mobile-phone terminal of the present embodiment as mentioned above, when a user touches the convex-shaped face of the touch panel of the deformation section 41 with a finger etc., the fact that the user has touched the position, i.e., the display position of the virtual key, can be recognized with the use of a touch detection signal from the touch panel concerned. Accordingly, the user is allowed to input or select information currently assigned to the virtual key concerned.
Explanation of the constitution for performing the discharge and suction control of the pump 43, the open/close control of the micro valve, and process of information inputting and selection, etc. based on a touch detection signal of the touch panel will be described later.
[Constitution for Realizing a Tactile and Visual Shape Change]
In the case of the basic constitution described above with reference to
In the mobile-phone terminal 50 according to an embodiment of the present invention, the tactile-visual UI panel 55 possesses at least two panel devices, the first layer and the second layer, and a pump 53.
The panel device of the first layer possesses the almost same components as the transparent sheet 42, the plural deformation sections 41, and the fluid channel 44 of the tactile-visual UI panel 45 explained above with reference to
In the tactile-visual UI panel 55 of the present embodiment, a panel device of the second layer is arranged over the front face of the transparent sheet 42 included in the first layer, and a touch panel (not shown) is arranged over the front face of the second layer concerned.
That is, in the tactile-visual UI panel 55 of the present embodiment, the second layer possesses a transparent sheet 52, plural deformation sections 51, and a fluid channel 54 at least.
Similarly to the transparent sheet 42 of the first layer, the transparent sheet 52 of the second layer concerned is formed by transparent elastomeric material etc. possessing elasticity. The transparent sheet 52 is provided in the front side (the screen surface side) of the display panel 46 of the mobile-phone terminal 50 and in the front side of the panel device of the first layer, and is arranged so that the transparent sheet 52 may cover almost the whole surface of the screen surface of the display panel 46 concerned. In the case of the mobile-phone terminal of the present embodiment, a not-shown transparent touch panel, which is the same as described above, is arranged over the front side of the transparent sheet 52 of the second layer so that almost the whole surface of the transparent sheet 52 concerned may be covered.
Similarly to the fluid channel 44 of the first layer, the fluid channel 54 of the second layer is formed, for example, by a tube-like transparent member through which fluid (for example, a liquid, a gas, a gelatinous material, etc.) is movable. One end of the tube of each fluid channel 54 of the second layer is connected with a fluid discharge suction orifice of the pump 53, and the other end of the tube is connected with each deformation section 51. Similarly to the fluid channel 44 of the first layer, it is desirable that the fluid channel 54 of the second layer concerned is formed by material which possesses flexibility of a certain degree in the longitudinal direction and in the direction orthogonal to the longitudinal direction of the tube, but does not change at least with respect to the diameter (namely, it does not expand nor contract in the diameter direction). In
The pump 53 of the tactile-visual UI panel 55 of the present embodiment is, similarly to the pump 43, operable to perform the discharge and suction of fluid and employs a so-called diaphragm micro pump, as an example. However, in the example of
Each of the plural deformation sections 51 of the second layer is formed by a transparent elastomeric material and possesses predetermined shape (a square shape in the example of
In the case of the mobile-phone terminal of the present embodiment, the tactile-visual UI panel 55 possesses the first-layer panel device and the second-layer panel device which are provided over the screen surface of the display panel 46. The first-layer panel device is arranged over the surface of the display panel 46 possessing the hardness of a certain degree as described above, and the second-layer panel device is arranged over the front side of the first layer. Accordingly, when the fluid is injected only into the deformation section 41 of the first layer, for example, and the circular section of the deformation section 41 concerned expands, the tactile-visual UI panel 55 concerned will be in the state where the circular section of the deformation section 41 is deformed to a convex shape in the opposite side to the screen surface of the display panel 46 (namely, to the second layer side), as shown in
Also in the example of
In
As mentioned above, according to the basic constitution of the tactile-visual UI panel 55 in the mobile-phone terminal of the present embodiment as shown in
In the mobile-phone terminal of the present embodiment, for example, when an image of a circular virtual key is displayed on a display screen so as to correspond to the place where each of the deformation sections 41 of the first layer of the tactile-visual UI panel 55 is arranged, and at the same time, the fluid is injected to each of the deformation sections 41 of the first layer to deform each of the deformation sections 41 to a convex shape, the circular virtual key on the display screen will actually appear visually as a key of a convex shape to a user, and the user will be able to recognize the deformation sections 41 of the convex shape as actual keys, tactilely as well as visually, by touching the deformation sections 41 with a finger etc. Furthermore, for example, when an image of a square virtual key is displayed on a display screen so as to correspond to the place where each of the deformation sections 51 of the second layer of the tactile-visual UI panel 55 is arranged, and at the same time, the fluid is injected to each of the deformation sections 51 of the second layer to deform each of the deformation sections 51 to a convex shape, the square virtual key on the display screen will actually appear visually as a key of a square convex shape to a user, and the user will be able to recognize the deformation sections 51 of the square convex shape as actual keys, tactilely as well as visually, by touching the deformation sections 51 with a finger etc.
Namely, according to the present embodiment, by the tactile-visual UI panel 55, the information which can be conveyed to a user tactilely and visually will be three kinds: the first one for when each deformation section 41 of the first layer is expanded and deformed to a circular convex shape, the second one for when each deformation section 51 of the second layer is expanded and deformed to a square convex shape, and the third one for when both of the deformation section 41 of the first layer and the deformation section 51 of the second layer are shrunk and made almost flat. Therefore, the flexibility in changing the tactile and visual information by the tactile-visual UI panel 55 concerned is increased. Of course, when each of the deformation section 41 of the first layer and the deformation section 51 of the second layer are expanded at the same time to form a circular convex shape and a square convex shape, the combination of the circular convex shape and the square convex shape becomes available, leading to further increased flexibility.
Since the touch panel is provided over the surface of the tactile-visual UI panel 55 which possesses the first layer and the second layer, in the mobile-phone terminal of the present embodiment shown in
Explanation of the constitution for performing the discharge and suction control of the pump 53, the open/close control of the micro valve, and information inputting or selection process, etc. based on a touch detection signal of the touch panel will be described later.
Although the number of layers is set to only two layers, the first layer and the second layer, in the example of
Although the convex shape formed by injecting the fluid into a deformation section is described in the examples of
Since the deformation section 41 of the tactile-visual UI panel 45 is assumed to possess a predetermined shape (a circular shape) in the basic constitution shown in
In the mobile-phone terminal 60 of the present embodiment shown in
Similarly to each of the above-described examples, the transparent sheet 62 is formed by transparent elastomeric material etc. possessing elasticity. The transparent sheet 62 is provided in the front side (the screen surface side) of a display panel 46 of the mobile-phone terminal 60, and is arranged so that the transparent sheet 62 may cover almost the whole surface of the screen surface of the display panel 46 concerned. Also in the present embodiment similarly to each of the above-described examples, a transparent touch panel (not shown) is provided in the front side of the transparent sheet 62.
For example, the dot deformation section 61 according to the present embodiment is arranged in matrix, each dot deformation section 61 respectively possesses a predetermined shape (in the example of
The fluid channel 64 is formed by a tube-like transparent member through which fluid (a liquid, a gas, a gelatinous material, etc.) is movable. In the present embodiment, the fluid channel 64 is classified into plural fluid supply channels 64in for supplying each dot deformation section 61 with the fluid discharged by the pump 63, and plural fluid suction channel 64out for returning the fluid discharged from each dot deformation section 61 to the pump 63. Furthermore, in the present embodiment, the fluid supply channel 64in is arranged for example, one for every column of the dot deformation section 61 in the matrix arrangement. Similarly, the fluid suction channel 64out is also arranged one for every column of the dot deformation section 61 in the matrix arrangement. One piece of the fluid supply channel 64in arranged per one column of the matrix arrangement possesses plural bifurcating channels which branches to one for every row in the column concerned. Similarly, one piece of the fluid suction channel 64out arranged per one column of the matrix arrangement also possesses plural bifurcating channels which branches to one for every row in the column concerned. The end of each bifurcating channel of the fluid supply channel 64in arranged per one column of the matrix arrangement is connected correspondingly to the fluid injection orifice of each the dot deformation section 61 provided for every row in the one column concerned. On the other hand, the common end which is opposite to the bifurcating channel side is connected to the fluid discharge orifice of the pump 63. Similarly, the end of each the bifurcating channel of the fluid suction channel 64out arranged per one column of the matrix arrangement is connected correspondingly to the fluid discharge orifice of each dot deformation section 61 provided for every row in the one column concerned. On the other hand, the common end which is opposite to the bifurcating channel side is connected to the fluid suction orifice of the pump 63. Similarly to the above case, the fluid channels 64 is formed by the material which possesses the flexibility of a certain degree with respect to the longitudinal direction of the tube or with respect to the direction orthogonal to the longitudinal direction concerned, on the other hand, the material does not change at least with respect to the diameter (the material does not expand nor contract in the diameter direction). In the above explanation, the common end which is opposite to the bifurcating channel side of the plural fluid supply channels 64in is connected with the fluid discharge orifice of the pump 63, respectively, and the common end which is opposite to the bifurcating channel side of the plural fluid suction channels 64out is connected with the fluid suction orifice of the pump 63, respectively. However, if there is only one fluid discharge orifice of the pump 63 for example, the common end which is opposite to the bifurcating channel side of the plural fluid supply channels 64in may be united to one, and may be connected to one fluid discharge orifice of the pump 63 concerned. Similarly, if there is only one fluid suction orifice of the pump 63, the common end which is opposite to the bifurcating channel side of the plural fluid suction channels 64out may be united to one, and may be connected to one fluid suction orifice of the pump 63 concerned.
The pump 63 is a diaphragm micro pump, for example, which is operable to discharge and suck the fluid. A fluid discharge orifice of the pump 63 is connected to the fluid supply channel 64in, and the fluid suction orifice of the pump 63 is connected to the fluid suction channel 64out.
In the present embodiment, as shown in
By performing control to bring one desired dot deformation section 61 into an expansion state and to bring another desired dot deformation section 61 into an empty (plane) state among the dot deformation sections 61 arranged in matrix, the transparent sheet 62 overlying the dot deformation section 61 which are brought into the expansion state is raised in the direction separating from the screen surface of the display panel 46, and, on the other hand, the transparent sheet 62 overlying the other dot deformation section 61 which are brought into the empty (plane) state remains at the low position near the screen surface of the display panel 46.
That is, according to the mobile-phone terminal of the present embodiment, as shown in
In the case of the basic constitution using the matrix arrangement of the dot deformation section such as explained in
The tactile-visual UI panel 75 of the mobile-phone terminal 70 shown in
In the mobile-phone terminal 70 according to an embodiment of the present invention, the tactile-visual UI panel 75 possesses at least two panel devices, a first-layer panel device and a second-layer panel device.
The first-layer panel device overlying nearer the display panel 46 possesses the almost same components as the transparent sheet 62, plural dot deformation sections 61 arranged in matrix, plural fluid supply channels 64in, and plural fluid suction channels 64out of the tactile-visual UI panel 65 explained in
In the tactile-visual UI panel 75 of the present embodiment, the second-layer panel device is arranged overlying the front of the transparent sheet 62 constituting the first-layer panel device, and a touch panel (not shown) is arranged over the front of the second-layer panel device concerned. In the tactile-visual UI panel 75 of the present embodiment, the second-layer panel device concerned possesses the transparent sheet 62, plural dot deformation sections 61 arranged in matrix, plural fluid supply channels 64in, and plural fluid suction channels 64out, almost similarly to the first-layer panel device.
The pump 63 of the mobile-phone terminal 70 of the present embodiment is, same as in the example of
Also in the example illustrated in
The mobile-phone terminal 70 according to the present embodiment possesses structure of the first layer and the second layer, and is able to control to bring into an expansion state the desired dot deformation section 61 among the dot deformation sections 61 arranged in matrix in the each layer, and to bring into an empty (plane) state another desired dot deformation section 61. Accordingly, as illustrated in
Namely, according to the present embodiment, it is possible to control the expansion state and empty (plane) state of the desired dot deformation section 61 of the first and the second layer. In other words, it is possible to control the height due to the first and the second layer from the display panel surface to at least three levels. This fact means that the flexibility in changing the tactile-visual information which can be conveyed to a user can be raised rather than the example of constitution of
A specific example of a tactile-visual user interface which can be provided to a user by use of the tactile-visual UI panel of the mobile-phone terminal of each embodiment described above is explained in the following with reference to
That is, in
[Compensation of a Magnified Image Due to Expansion of a Deformation Section of a Tactile-Visual UI Panel]
When the deformation section of the tactile-visual UI panel is deformed to an expansion state to form a convex-shaped button, etc., as in the example of
In the exemplified constitution shown in
When the fluid is discharged to the fluid channel 84 in order to bring the deformation section 81 of the upper layer into an expansion state, the pump 83 operates so as to suck the fluid through the fluid channel 94 in order to bring the deformation section 91 of the lower layer into a contracted state, and at the same time, the pump 83 controls to adjust the fluid discharge amount to the deformation section 81 and the fluid suction amount from the deformation section 91, so that the curvature of the portion which deforms to a convex shape due to expansion by injection of the fluid into the deformation section 81 and the curvature of the portion which deforms to a concave shape due to contraction by discharge of the fluid from the deformation section 91 may become same. When the fluid is sucked from the fluid channel 84 in order to bring the deformation section 81 of the upper layer into a flat state, the pump 83 operates so as to discharge the fluid to the fluid channel 94 in order to bring the deformation section 91 of the lower layer into a flat state, and simultaneously, the pump 83 controls to adjust the fluid suction amount from the deformation section 81 and the fluid discharge amount to the deformation section 91, so that both the deformation section 81 and the deformation section 91 may be in a flat state.
Therefore, according to the constitution of
As another example for compensating the state where an image will appear as enlarged due to the deformation section deformed to a convex shape, the display image of the display panel may be reduced at the reduction ratio corresponding to the magnification ratio of the image due to the deformation section deformed to the convex shape. In this case, the lower layer becomes unnecessary.
[Internal Constitution Overview of a Mobile-Phone Terminal]
In
A controller 10 possesses a CPU and performs control of communication in the communication circuit 11, signal processing and its control, image processing and its control, control of non-contact communication, and other various signal processing and control of every unit. The controller 10 also performs execution of various kinds of application programs stored in a program memory 16 of a memory unit 15.
A speaker 21 is used for a receiver speaker provided in the mobile-phone terminal, or for an outputting speaker of a ringer tone, an alarm sound, warning beep, a reproduced music, a digital voice, and a voice accompanying a reproduced moving image and the like. The speaker 21 converts the sound signal supplied from a sound signal processor 20 into an acoustic wave, and emits the acoustic wave into the air.
A microphone 22 is used for transmission of a voice and for collecting an external voice. The microphone 22 converts an acoustic wave into a sound signal, and sends the sound signal to the sound signal processor 20.
The sound signal processor 20 performs predetermined sound signal processing, such as decompression/decoding and error correction of the audio data which is processed for compression/encoding and inputted via a data line. The sound signal processor 20 D/A-converts the processed audio data to a sound signal, amplifies the sound signal, and outputs the amplified sound signal to the speaker 21. The sound signal processor 20 amplifies and A/D-converts the input voice signal supplied from the microphone 22, performs compression/encoding and addition of error correcting code, etc., if needed, to the A/D-converted audio data, and outputs the audio data via the data line.
A display unit 23 includes a display device such as a liquid crystal display panel, and a display driving circuit of the display panel, for example. The display unit 23 displays a character, various messages, a static image, a dynamic image, etc. on the screen of the display panel.
An operating input unit 24 corresponds to the transparent touch panel described above in the case of the mobile-phone terminal of the present embodiment. That is, in the operating input unit 24, when a user touches the surface of a transparent touch panel with a fingertip, a stylus pen, etc., for example, the contact is detected and the detection signal is outputted to the controller 10 through the data line or a control line. Accordingly, the controller 10 can recognize which place of the transparent touch panel is touched by the user (the input position is recognized), and can perform processing corresponding to the input position. As an example, in the case of the example of
A deformation section 25 is the deformation section of the tactile-visual UI panel of the present embodiment described above, a micro valve 26 is the micro valve of the tactile-visual UI panel of the present embodiment, and a pump 27 is the pump described above.
A pump control unit 28 controls operation of fluid discharging and fluid suction in the pump 27 based on the control data from the controller 10. Namely, the control data from the controller 10 includes a control value of fluid discharge amount when the pump control unit 28 controls the fluid discharging operation of the pump 27 and a control value of fluid suction amount when the control unit 28 controls the fluid suction operation of the pump 27. When the tactile-visual UI panel of the mobile-phone terminal of the present embodiment is provided with plural fluid channels, the control data also includes a control value indicating which fluid channel the fluid is to be discharged to or the fluid is to be sucked from.
When the tactile-visual UI panel of the mobile-phone terminal of the present embodiment is provided with plural micro valves, a valve control unit 29 performs a valve opening control and a valve closing control of each micro valve, based on the control data from the controller 10. Namely, the control data from the controller 10 includes a control value to be used by the valve control unit 29 in performing the valve opening control and the valve closing control of each micro valve 26, and also includes a control value for adjusting the amount of valve opening or the amount of valve closing, if the micro valve of the present embodiment is able to adjust the amount of valve opening or the amount of valve closing.
The memory unit 15 includes ROM (Read Only Memory) and RAM (Random Access Memory). ROM includes a rewritable storage medium such as a NAND-type flash memory, and stores for example, in the program memory 16, a program of OS (Operating System), a control program for the controller 10 to control each unit, a control program for controlling the pump and the valve in the present embodiment, and various kinds of application programs. ROM also stores compressed/encoded music data contents or compressed/encoded moving image data contents, various kinds of initial setting values, font data, dictionary data, machine name information, terminal identification information, and the like. RAM stores data as required, serving as a working area at the time of the controller 10 performing various kinds of data processing. A removable external memory cards may be included in the memory unit 15.
An image processor 13 generates image data, such as a standby screen and an E-mail sentence, an address book, various user interface screens, and a reproduced moving image of moving image data contents, and a received moving image of digital television broadcasting. The image processor 13 sends the image data to display on the display panel of the display unit 23 via the data line. The image processor 13 possesses a drawing engine 14, and generates an image data by a rendering process. The image data generated with the drawing engine 14 concerned is also sent and displayed on the display panel of the display unit 23. As mentioned above, processing to display a virtual button image on the display screen and to attach shading to the button image can be also performed by the drawing engine 14. About the rendering process or shading attachment to an image, the existing computer graphics art can be applied; therefore, the detailed explanation thereof is omitted here.
When the tactile-visual UI panel possessing the multi-layer structure of the panel device provided with plural dot deformation sections in matrix arrangement is employed, and when, for example, as illustrated in
By the rendering process in the drawing engine 14, the image processor 13 of the mobile-phone terminal of the present embodiment generates the image data for performing the two-dimensional display of the virtual three-dimensional image as indicated in
Using the three-dimensional graphics data, the controller 10 calculates the height in the virtual three-dimensional space concerned for every polygon drawn by the computer graphics technology. The controller 10 sets up a split screen area which splits the display screen into matrices, and collects the height information in the virtual three-dimensional space of each polygon which constitutes the images to be displayed in each split screen area (the images that the three-dimensional image is divided for every split screen area), so as to correspond to each dot deformation section arranged in matrix on the tactile-visual UI panel. The controller 10 sets up the information possessing the highest value among the height information of the polygon in the split screen area concerned as the height information of the virtual three-dimensional image in the split screen area. The controller 10 generates the height information with respect to the perpendicular direction to the display panel screen surface from the height information of the virtual three-dimensional image in each split screen area, and generates the control data to respectively perform deformation control for each dot deformation section arranged in matrix in the tactile-visual UI panel, from the height information with respect to the perpendicular direction to the display panel screen surface concerned.
To give the explanation more concretely, the controller 10 generates a control data which brings to an empty (plane) state the whole dot deformation sections of each layer in the tactile-visual UI panel, at the position corresponding to the split screen area which possesses the image exhibiting the lowest height in the virtual three-dimensional space among the virtual three-dimensional image in the split screen area. The controller 10 also generates a control data which brings to an expansion state the whole dot deformation sections of each layer in the tactile-visual UI panel at the position corresponding to the split screen area which possesses the image exhibiting the highest height in the virtual three-dimensional space among the virtual three-dimensional image in the split screen area. As for the other dot deformation sections of each layer in the tactile-visual UI panel at the position corresponding to the remaining split screen area excluding the split screen area which possesses the image exhibiting the highest height in the virtual three-dimensional space and the split screen area which possesses the image exhibiting the lowest height in the virtual three-dimensional space among the virtual three-dimensional image in the split screen area, the controller 10 generates a control data which brings to an expansion or an empty (plane) state the dot deformation section of each layer in the tactile-visual UI panel at the position corresponding to the split screen area concerned, corresponding to the height of the virtual three-dimensional image in the split screen area. The controller 10 controls the pump control unit 28 and the valve control unit 29 by the control data generated above. Accordingly, as explained in
In addition, although the illustration is omitted in
As explained above, in the mobile-phone terminal according to an embodiment of the present invention, it is possible to realize a user interface which can convey information in a tactile and visual mode to a user, by performing the deformation control of a tactile-visual UI panel, and especially it is possible to realize a user interface which can change the information conveyed in a tactile mode as well as the information conveyed in a visual mode, by forming the tactile-visual UI panel in multi-layer structure, and performing deformation control for every layer.
In the mobile-phone terminal of the present embodiment, ultimate thinness of the mobile-phone terminal is realizable, because any mechanically-structured button or key is not included.
Explanation of the embodiments of the present invention described above is an example of the present invention. Therefore, the present invention is not limited to the embodiments described above, and various modifications and alterations may be possible depending on design requirements and other factors insofar as they do not deviate from the technical idea concerning the present invention.
The user interface device and the personal digital assistant according to an embodiment of the present invention can be applied not only to a mobile-phone terminal, but to PDA (Personal Digital Assistant) and a personal computer, a portable video game device, a portable digital television receiver, a car navigation system, other various terminals, and in addition, to a stationary apparatus.
In the embodiment described above, the example in which a transparent touch panel is provided over the upper surface of the tactile-visual UI panel is illustrated. One example of the transparent touch panel is a touch panel of the so-called capacitance detection system. Of course, the present invention is not limited to the example. The present invention is applicable to, for example, a touch panel of the method which detects change of resistance by depression pressure such as the so-called resistance wire type, an optical type touch panel which detects a light interrupted with a finger etc. or a light reflected with a finger, etc., a magnetic induction type touch panel which detects change of magnetism using a dedicated pen such as a stylus pen, and other touch panels. The transparent touch panel may be provided between the display panel and the tactile-visual UI panel, i.e., the underface side of the tactile-visual UI panel. When the transparent touch panel is especially provided in the underface side of a tactile-visual UI panel, it becomes unnecessary for the transparent touch panel concerned to possess flexibility as needed in the above-described embodiment. When the transparent touch panel of the method which can detect the external pressure applied with a finger etc. is employed, the pressure variation by expansion, etc. of the deformation section of the tactile-visual UI panel may be accidentally detected as if the external pressure with a finger etc. were applied. However, in such a case, it may be possible, as an example, to judge the difference between the pressure value by expansion of the deformation section and the pressure value by external pressure with a finger etc. and to avoid the misjudgment to accidentally detect the pressure value by the expansion of the deformation section as the external pressure with the finger etc.
In addition, the user interface device of the present embodiment may be formed not only in a compact apparatus such as a personal digital assistant, but also with a large-sized display or screen and a user interface panel (UI panel) arranged over the upper surface of the large-sized display or screen. The large-sized display or screen includes, for example, a super-large screen display panel arranged on the roof of a building, on an outer wall surface, or on an indoor inner wall surface, a large screen of a stadium, and a large screen display panel arranged on the ground of recreation facilities etc. In the case of the large-sized user interface device provided with such a large screen display panel of size of from tens of inches to hundreds of inches or beyond, the concavo-convex part formed by the deformation section of the UI panel may possess a width in scale of from several centimeters to several meters, for example, and a height in scale of from several centimeters to several meters. It is desirable to set the hardness of the deformation section to a desired degree of hardness, corresponding to the size of the user interface device concerned.
It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.
| Number | Date | Country | Kind |
|---|---|---|---|
| P2007-233369 | Sep 2007 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 6535201 | Cooper et al. | Mar 2003 | B1 |
| 6897151 | Winter et al. | May 2005 | B2 |
| 7352356 | Roberts et al. | Apr 2008 | B2 |
| 20090002328 | Ullrich et al. | Jan 2009 | A1 |
| Number | Date | Country |
|---|---|---|
| 6 259006 | Sep 1994 | JP |
| 10 27053 | Jan 1998 | JP |
| 10 134671 | May 1998 | JP |
| 10 214158 | Aug 1998 | JP |
| 10 255106 | Sep 1998 | JP |
| 11 15599 | Jan 1999 | JP |
| 2000 49913 | Feb 2000 | JP |
| 2000 148393 | May 2000 | JP |
| 2003 91233 | Mar 2003 | JP |
| 2004 157933 | Jun 2004 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20090066672 A1 | Mar 2009 | US |
