The present disclosure relates to an information processing apparatus, an information processing method, and a program and, more particularly, to an information processing apparatus, an information processing method, and a program that are configured to control the display of objects of a user's interest.
Recently, stereoscopic display technologies for three-dimensionally displaying content in a virtual three-dimensional space have been gaining popularity (refer to Japanese Patent Laid-open No. Hei 08-1.16556 below, for example). In the case of displaying an object in a virtual three-dimensional space, the adjustment in parallax amounts between an image for the right eye and an image for the left eye for presenting an object on a stereoscopic display for the left and the right eye of a viewer allows the presentation of the object at any distance in the depth direction from an extremely close range before the eyes of the viewer to an infinite direction.
However, when a person looks at a stereoscopic display, increasing the parallax of an object displayed on the display too much takes long for the person to focus the eyes on the object, thereby causing a problem of straining the eyes to lead to eye fatigue. In order to overcome this problem, a method is proposed in which a range of the projection and retracting amounts of an object is restricted, thereby mitigating the eye fatigue and the sense of unnaturalness.
Especially, in a situation where, as with small-size displays designed for portable devices, an object is stereoscopically displayed at an arm's length for example, viewing distance is extremely short, the above-mentioned problem therefore presents itself more conspicuously, thereby making it difficult to stereoscopically present the object beyond several centimeters from the front and back of the display. In order to circumvent this problem, a method is proposed in which the parallax amount of an image is decreased more especially in small-size displays than in large-size displays with long viewing distance, thereby mitigating the eye fatigue and the sense of unnaturalness.
Generally, the retracting presentation from the screen, rather than the projecting presentation from the screen, is advantageous in the mitigation of the eye fatigue and the sense of unnaturalness and does not strain the eyes of the viewer, thereby providing comfortable browsing of stereoscopically expressed objects. Hence, a method is proposed in which a retracting amount is set higher than a projection amount to execute stereoscopic display that provides the sense of depth.
However, while the eye fatigue can be mitigated by decreasing the projection amount or retracting amount of an object, stereoscopic effects are reduced, thereby lacking the sense of realism or an impact. Therefore, the development of technologies has been desired for widening the range of the projection amount and retracting amount of an object by developing a new method of facilitating for each viewer to set the focus of the eyes on a desired object, thereby providing the presentation having the sense of realism and an impact.
Therefore, the present disclosure has been made in order to solve the problems described above. It is desirable to provide a new and improved information processing apparatus, an information processing method, and a program that are configured, in accordance with the attention of a user to a particular object, to execute animation display of this particular object in the depth direction.
In carrying out the disclosure and according to one embodiment thereof, there is provided an information processing apparatus including an acquisition block configured to acquire a detection result associated with attention of a user to an object displayed on a display screen capable of stereoscopically displaying an image, and a display control block configured to animation-display the object in the direction of depth in accordance with the user attention on the basis of the acquired detection result.
In carrying out the disclosure and according to one embodiment thereof, there is provided an information processing method including acquiring a detection result associated with a user attention to an object displayed on a display screen capable of stereoscopically displaying an image, and animation-displaying the object in the direction of depth in accordance with the user attention on the basis of the acquired detection result.
In carrying out the disclosure and according to one embodiment thereof, there is provided a program for causing a computer to execute processing of acquiring a detection result associated with a user attention to an object displayed on a display screen capable of stereoscopically displaying an image, and processing of animation-displaying the object in the direction of depth in accordance with the user attention on the basis of the acquired detection result.
As described above and according to the invention, in response to user's attention to a particular object, said particular object can be animation-displayed in the depth direction.
The embodiment of the present disclosure will be described in further detail below with reference to the accompanying drawings. It should be noted that, in the present specification and drawings attached thereto, component elements having substantially the same functional configuration are denoted by the same reference numeral and the description of these component elements is omitted.
The description of the embodiment will be made in the following order:
(1-1) A configuration of an information processing apparatus;
(1-2) Operations of the information processing apparatus;
(1-3) Adjustment of parallax amounts;
(1-4) Switching between screens; and
(1-5) Variations
(1-1) A Configuration of an Information Processing Apparatus
The following describes a configuration of an information processing apparatus according to one embodiment of the present disclosure with reference to
As shown in
The display screen 120 is a display device such as LCD (Liquid Crystal Display), PDP (Plasma Display Panel), or organic EL (Electro-Luminescence) panel, for example. The display screen 120 is a display capable of stereoscopic display. The display screen 120 executes stereoscopic display by a method in which a pair of polarized glasses for example is used to allow a viewer to see a view point image (or a parallax image) based on the polarized states different for the eyes of the viewer. Any other methods are available including one in which, without using a pair of polarized glasses, a predetermined view point image of two or more view point images is introduced into the eye balls of the viewer through a parallax barrier or a lenticular lens, and another that is known as frame sequential. It should be noted that, as shown in the figure, the display screen 120 may be unitized with the information processing apparatus 100, and arranged on the surface of the apparatus. Alternatively, the display screen 120 may be arranged independently of the information processing apparatus 100.
The virtual three-dimensional space Sv is displayed on the display screen 120. In the embodiment, three directions shown in the figure are defined in the virtual three-dimensional space Sv; the x-axis direction, the y-axis direction, and the z-axis direction. The x-axis direction is indicative of the left and right directions of the display screen 120. The y-axis direction is indicative of the up and down directions of the display screen 120. The z-axis direction is indicative of the depth direction of the display screen 120.
In the virtual three-dimensional space Sv, two or more objects 200a through 200g are displayed. The objects may include the point object 200g that is operated by the user and content objects 200a through 200f, for example. It should be noted that the animation display of objects in the embodiment will be described later.
The acquisition block 130 gets the results of the detection associated with the attention of the user to objects displayed on the screen. In
In order to get the detection results of user attention, the acquisition block 130 may have a camera or an infrared radiation emission block and an infrared radiation reception block, for example. In the embodiment, the acquisition block 130 having a camera is arranged on such a location that facilitates the detection of a viewer's face, such as the top of the information processing apparatus 100, for example.
Attention of User
In a method of detecting whether the user is paying attention to a particular object in the embodiment, face detection (or face recognition) is executed from the image data obtained by use of the camera of the acquisition block 130, thereby estimating whether the user is paying attention to a particular object by the presence or absence of the user's face.
Various other methods are available for methods of detecting whether the user is paying attention to a particular object. For example, the presence or absence of the user's face and the direction of the user's face may be detected by use of the above-mentioned face recognition technology, thereby estimating a particular object to which the user is paying attention on the basis of the detection. On the basis of the detection results of the direction of the user's face and the distance between the display screen 120 and the user's face, a particular object to which the user is paying attention can be estimated. By detecting that there are two or more eyes, a particular object to which the user is paying attention can be estimated.
By detecting a reflected light of infrared radiation from the retina of the user's eyes, the attention of the user may be estimated. A sensor, such as a camera, may be attached to the eyeglasses of the user to detect the distance and directional relation between the eyeglasses and the display screen 120, thereby estimating the attention of the user.
From an operational state of a user's device, the attention of the user may be estimated. Especially, if the information processing apparatus 100 is a mobile device, such as a mobile phone, a portable terminal, or a portable game machine, the attention of the user may be estimated on the basis of the state of an operation executed on the user's device or the directivity of sound emitted from the device. In addition, the attention of the user may be estimated by any other methods as long as objects to which the user is paying attention can be estimated on the basis of eye contact, sight line movement, sight line direction, presence or absence of the body, the direction of the body, the direction of the head, user's gesture, or brain signals, for example.
As described above, the detection results associated with user attention may include at least one or more of the presence or absence of the face, the direction of the face, the distance between the above-mentioned display screen and the face, two or more eyes, the infrared light reflected from the user's eyes, operation of the device, the directivity of the sound emitted from the device, brain signals, eye contact, sight line movement, sight line direction, the presence or absence of the body, the direction of the body, the direction of the head, and user's gesture.
The acquisition block 130 may be arranged at such a location easy to detect viewer's face as the top of the information processing apparatus 100 and in the orientation corresponding to the orientation of the display screen 120, as shown in
The information processing apparatus 100 shown in
Referring to
The image input block 140 is used to enter an image to be displayed on the display screen 120. The display control block 150 adjusts a parallax amount in accordance with user's attention to a particular object. For example, if user's attention departs from a particular object, the display control block 150 accordingly decreases the parallax amount of the object. If user's attention is directed to a particular object, the display control block 150 accordingly increases the parallax amount of the object.
Methods of adjusting the parallax amount of a particular object include one that the parallax amount is adjusted by simply adjusting the depth amount of content to the depth direction (the Z direction), and another that the parallax (or the angle of convergence) is actually adjusted. For example, in order to realize animation display on the display based on binocular parallax, two or more intermediate parallax images having different parallax amounts (or offset amounts) of a right-eye image 200eR and a left-eye image 200eL in the form of supplementing the parallax amounts in the final states of the right-eye image 200eR and the left-eye image 200eL. Next, the intermediate parallax images are displayed (the center diagram of
The display control block 150 displays the objects 200a through 200g to be arranged in virtual three-dimensional space Sv onto the display screen 120 as stereoscopic images. On the basis of the adjustment of parallax amount, the display control block 150 animation-displays a particular object in the depth direction. To be more specific, as the user's attention departs from a particular object, the display control block 150 gradually decreases the parallax amount of the object accordingly. Consequently, the display control block 150 can animation-display the particular object so as to decrease the depth amount of the object. When user's attention is directed to a particular object, the display control block 150 gradually increases the parallax amount of the object accordingly. Consequently, the display control, block 150 can animation-display the particular object so as to increase the depth amount of the object.
The following describes animation display with reference to
For example, the left diagrams of
Accordingly, an object is gradually moved in the depth direction to a location at which the object is actually displayed, so that the user can easily stepwise adjust the eye focus to make the eye focus follow the object. Consequently, as compared with the case where animation display is not executed, the range of depth amounts can be widened, thereby realizing realistic stereoscopic display. It should be noted that the depth amount of an object denotes an amount of projection forward from the display screen 120 or an amount of retraction backward from the display screen 120.
In
The display control block 150 is a computation processing apparatus for executing display operations and can be realized by a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), or a DSP (Digital Signal Processor), for example. The display control block 150 may operate as instructed by programs stored in the storage block 160.
It should be noted that programs that realize the functions of the display control block 150 may be provided to the information processing apparatus 100 as stored in a removable storage medium, such as a disc recording medium or a memory card, or the programs may be downloaded to the information processing apparatus 100 through a LAN (Local Area Network), the Internet, or other networks, for example.
The storage block 160 stores data necessary for the processing to be executed by the information processing apparatus 100. In this first embodiment, the storage block 160 also stores a first predetermined time and a second predetermined time that are predetermined as the threshold values for animation display. The storage block 160 may be a storage apparatus, such as a RAM (Random Access Memory) or a ROM (Read Only Memory). Alternatively, the storage block 160 may be a removable storage medium, such as an optical disc, a magnetic disc, or a semiconductor memory, and may be a combination of a storage apparatus and a removable storage medium. The storage block 160 may store programs for realizing the functions of the display control block 150, for example by making the CPU or the DSP execute these functions.
(1-2) Operations of the Information Processing Apparatus
The following describes animation display processing according to the embodiment of the disclosure with reference to
First, in step S305, the image input block 140 enters an image that includes two or more objects 200a through 200g. In step S310, the display control block 150 two-dimensionally displays the input image on the display screen 120 in a state where there is no parallax. In step S315, the acquisition block 130 gets detection results associated with user attention to a particular object. The display control block 150 determines the user's attention to a particular object. If the user is found not paying attention to a particular object, then the procedure returns to step S310 to repeat step S310 and step S315 until the user pays attention to a particular object.
If the user is found paying attention to a particular object, then the procedure goes to step S320, in which the display control block 150 animation-displays the particular object to which the user is paying attention such that the depth amount of this object is enlarged (the parallax enlarged animation display).
Consequently, the sign plus or minus attached to a parallax amount to be adjusted indicates that the object 200e gradually projects forward from the display screen 120 from a display state in which there is no parallax, and is animation-displayed up to a target stereoscopic display location as shown in the upper part of
Next, in step S325, the display control block 150 displays the stereoscopic image of the particular object at a target stereoscopic display location. In step S330, the display control block 150 determines whether the user is paying attention to the particular object. If the user is found paying attention to the particular object, then the procedure returns to step S325 to repeat step S325 and step S330 to display the stereoscopic image of the particular object at the target stereoscopic display location until the user averts the eyes from the particular object.
If the user is found to have averted the eyes from the particular object, then the procedure goes to step S335, in which the display control block 150 determines how long the user has averted the eyes. If this averted time is found to be equal to or longer than the predetermined threshold value A (equivalent to the first predetermined time) of the eye averted time, then the procedure returns to step S325, in which the display control block 150 displays the stereoscopic image of the particular object at the target stereoscopic display location, step S325 through step S335 are repeated until the eye averted time gets longer than the threshold value A.
If the eye averted time gets longer than the predetermined threshold value A, then the procedure goes to step S340, in which the display control block 150 animation-displays the particular object such that the depth amount of the particular object is reduced (the parallax reduced animation display). Next, in step S345, the display control block 150 determines whether the user is paying attention to the particular object. If the user is found paying attention to the particular object, then the procedure returns to step S320, in which the display control block 150 animation-displays the particular object such that the depth amount of the particular object is enlarged. Then, the display control block 150 executes the processing of steps S325 onward again.
If the user is found not paying attention to the particular object in step S345, then the procedure returns to step S335, in which the display control block 150 continues the parallax reduced animation display in a duration longer than threshold value A of averted eye time and shorter than a threshold value B (equivalent to the second predetermined time); if the eye averted time gets equal to or longer than the threshold value B in step S335, then the procedure goes to step S310, in which the display control block 150 displays the image of the particular object on the display screen 120 in the state in which there is no parallax. Then, the display control block 150 executes the processing of steps S315 onward again.
According to the processing described above, an object is gradually moved in the direction of depth by animation display, so that the user can gradually adjust the eye focus in accordance with the animation display, thereby making the eye focus follow the object. Therefore, as compared with the case where animation display is not executed, the range of object's projection amount and retracting amount can be widened to execute stereoscopic display that is realistic and powerful.
If the duration in which the user averts eyes from the particular object is equal to or shorter than the threshold value A, then the procedure returns to step S325, in which the stereoscopic image of the particular object is displayed on the screen. This is because the user can easily focus on a stereoscopically displayed object without starting animation display after a very short eye averted time. If animation display is started after a very short eye averted time, display switching occurs frequently even when the user returns the eyes to that object immediately after the very short eye averted time, thereby complicating the processing.
If the eye averted time is longer than the threshold value A and shorter than the threshold value B, the parallax reduced animation display is executed in step S340 as described above. This is because the object is gradually moved from the target stereoscopic display location in the direction in which the depth amount is reduced by animation display, so that, if the user returns the eyes to the object, the user can easily adjust the eye focus in accordance with the animation display.
If the eye averted time is equal to or longer than the threshold value B, the procedure returns to step S310 to display the image (or the object) on the screen in the state in which there is no parallax. This is because, if the user averts the eyes from the object for long, displaying the object without parallax most removes the fatigue from the user's eyes, thereby providing comfortable browsing.
In step S320, the display control block 150 executes the parallax enlarged animation display and then displays the stereoscopic image of the object onto the screen in step S325. This is because connecting the object in animation display from the displaying of the object on the screen without parallax to the target stereoscopic display location reduces unnaturalness of display so that the user can easily view the display, thereby minimizing the eye fatigue of the user.
It should be noted that, in step S335, a method of displaying the image onto the screen is determined on the basis of the length of the eye averted time. However, in step S335, this display may be controlled on the basis of the parallax amount of the object instead of the length of eye averted time. For example, in the case of image content having a large parallax amount, such as an excessively projecting object or an excessively retracting object, after averting the eyes for a while, then the procedure returns to step S310, in which the image content may be displayed on the screen without parallax. If the parallax amount is small, after averting the eyes for a while, then the procedure returns to step S325, in which the stereoscopic display may be continued without changing parallaxes. If the parallax amount is neither large nor small, the procedure may go to step S340 to execute the parallax reduced animation display. It should be noted that the condition of averting the eyes for while and the condition that the parallax amount is neither large nor small may be predetermined and the values of these conditions may be stored in the storage block 160 in advance.
(1-3) Adjustment of Parallax Amounts
As described above, the display control block 150 adjusts the parallax amount of a particular object in accordance with user's attention on the basis of detection results acquired by the acquisition block 130 and animation-displays the object corresponding to the adjusted parallax amount.
At this time, the display control block 150 can adjust the parallax amount of an object in a linear or nonlinear manner. Consequently, the speed of animation display can be controlled. For example, the graph shown in the upper part of
The graph in the upper left shows a method of executing adjustment to linearly reduce the absolute value of a parallax amount during time from the starting of animation display to the ending thereof. According to this method, the parallax reduced animation display is executed for gradually reducing object projecting representation or retracting representation. In order to mitigate user's fatigue and the sense of unnaturalness, the retracting representation into the depth from the display screen 120 is generally easier than the projecting representation from the display screen 120 for the user's eyes to browse the screen. Hence, in each graph shown in
The graph in the upper right is indicative of a method of linearly increasing the absolute value of a parallax amount. According to this method, the parallax enlarged animation display is executed for gradually enlarging object's projecting representation or retracting representation. In this case too, the speed of animation display to be executed behind the display screen 120 is greater than the speed of animation display to be executed in front of the display screen 120.
On the other hand, the graph in the lower left executes adjustment such that the absolute value of a parallax amount gets linearly small during the time from the start and end of animation display. This adjustment allows the execution of the parallax reduced animation display in which the object projecting or retracting speed changes temporally. As described above, the retracting representation into the depth from the display screen 120 is generally easier than the projecting representation from the display screen 120 for the user's eyes to browse the screen. In addition, at animation start time to and end time t: and in time zones in the vicinity thereof, it is harder for the human eyes to follow an object than in a time zone therebetween. In consideration of this parallax state of the user, the graph in the lower left decreases the variable of the parallax amounts at animation start time to and end time t; and in the time zones in the vicinity thereof as compared with the variable in other time zones, thereby executing animation display slowly at the animation start time and end time.
Likewise, the graph in the lower right linearly adjusts the absolute value of a parallax amount to a great degree. According to this method, the parallax enlarged animation display in which the speed of object projection or retraction is varied can be executed. In this case too, the changing speed of animation display is lower at the start and end thereof than halfway in therebetween, so that the load on the user's eyes is mitigated to make easier for the user's eyes to follow the object. Hence, the range of the depth amount of an object can be further widened, thereby realizing the stereoscopic display that is more real and powerful.
(1-4) Switching Between Screens
The animation display method described above (with reference to
For example, step S705 shows a state in which an input image is stereoscopically represented (a stereoscopic screen A displayed state). This is equivalent to step S325 shown in
In step S720, when a new image is entered for example, screen A is switched to screen B. In step S725, the image of screen B switched is displayed without parallax. This is equivalent to step S305 and step S310 shown in
As described above, when executing stereoscopic display for each of the images that are displayed by switching the screens, an intermediate image that is not the target stereoscopic display for each image is displayed in the process of display up to the target stereoscopic display. As a result, each image can be smoothly transitioned from the non-stereoscopic display easy for the eyes to the target stereoscopic display in the form of animation, thereby mitigating the load on the user's eyes when the user see stereoscopic display screens.
As described above, according to the information processing apparatus 100 according to the embodiment of the disclosure, in response to the user's attention to a particular object, the object is animation-displayed in the depth direction. This configuration widens the range of object's projecting amount and retracting amount, thereby executing stereoscopic display that is realistic and powerful.
Especially, in a situation where an object is stereoscopically displayed at an arm's length distance for example on small-sized displays installed on portable devices, the viewing distance is very short, so that the load on user's eyes is very high, thereby increasing eye fatigue as compared with the case of large-sized displays. Therefore, attempting to present an object beyond several centimeters from the front and back of the display makes it difficult to provide stereoscopic display. However, according to the animation display in the depth direction according to the present embodiment, the range of object's projecting amount and retracting amount can be widened while mitigating the eye fatigue and the sense of unnaturalness even on small-sized displays, thereby executing stereoscopic display that is more realistic and powerful.
(1-5) Variation
The animation display to be executed by the information processing apparatus 100 is applicable to other uses. A variation in the present embodiment is described as one of the examples, in which animation display is used when changing image display methods between a normal view area in which stereoscopic viewing can be done and a reverse view area in which stereoscopic viewing is difficult.
First, it is determined whether an area is viewable from a user's viewing location. For example, a user's viewing location is detected by use of camera or the like and a result of the detection is transmitted to the acquisition block 130. On the basis of the detection result obtained by the acquisition block 130, the display control block 150 animation-displays an image in the depth direction in accordance with the user's viewing location. To be more specific, it is determined whether the user's viewing location is in the normal view area or the reverse view area. When the user's viewing location has shifted from the normal view area to the reverse view area, the parallax reduced animation display is executed. In doing so, the image may be transitioned until the parallax reduced animation display is executed without parallax or the image may be transitioned until the stereoscopic display with the depth adjusted for that user is executed.
According to the above-mentioned variation, the boundary between the normal view area and the reverse view area can be linked by executing the parallax reduced animation (or fade animation) at the timing described above. This configuration mitigates the uncomfortableness of eyes that occurs at the boundary between the normal view area and the reverse view area.
According to the embodiment and variation described above, in response to the user's attention to a particular object, this object can be animation-displayed in the depth direction, thereby widening the range of the projecting amount and retracting amount of the object. In addition, the above-mentioned novel configuration facilitates the browsing by the user, the fine adjustment of the degrees of projection and retraction in stereoscopic display need not be done for each screen and content, thereby facilitating the screen design of the graphical user interface (GUI) of stereoscopic display.
In the embodiment and the variation described above, the operations of component blocks are related with each other and, by considering the interrelations thereof, the component blocks can be replaced by a sequence of operations and a sequence of processing. Consequently, the embodiments of the information processing apparatus can be provided as embodiments of an information processing method and embodiments of programs for realizing the functions of the information processing apparatus by use of a computer.
While preferred embodiments of the present disclosure have been described with reference to the accompanying drawings, such description is for illustrative purpose only, and it is to be understood that changes and variations may be made without departing from the spirit or scope of the following claims.
For example, in the embodiment described above, an object is animation-displayed in the depth direction in accordance with the user's attention by assuming that the number of users who view is one; however, the present disclosure is not limited to this example. For example, if there are two or more viewing users, the acquisition block may acquire the detection results associated with users' attentions for each user. In addition, the display control block may animation-display objects in the depth direction in response to the users' attentions for each user on the basis of the acquired detection results. It should be noted that, in this example, control is executed such that the positional relation in the depth direction of two or more objects displayed on the screen is maintained.
It is also practicable, when a user intentionally gestures, to execute animation display accordingly. In doing so, the acquisition block acquires an operation done by the user, and the display control block readjusts the parallax amount in accordance with the acquired user operation and animation-displays an object on the basis of the readjusted parallax amount. According to this configuration, if the user intentionally gestures during the execution of the parallax reduced animation display, for example, the display method is switched to the parallax enlarged animation display in accordance with the gesture, thereby displaying the object as if the object returns. In addition, animation display can be started for the first time by an operation done by the user. Animation display can also be terminated by an operation done by the user.
Number | Date | Country | Kind |
---|---|---|---|
2010-284583 | Dec 2010 | JP | national |
This application is a continuation of U.S. patent application Ser. No. 13/314,897 (filed on Dec. 8, 2011), which claims priority to Japanese Patent Application No. 2010-284583 (filed on Dec. 21, 2010), all of which are hereby incorporated by reference in their entirety.
Number | Date | Country | |
---|---|---|---|
Parent | 13314897 | Dec 2011 | US |
Child | 15609765 | US |