Patent Application
20060238502
(1) Field of the Invention
The present invention relates to an image display and image reproduction technology, and particularly relates to an image display device and an image display method for displaying a textually rich image of an image of a subject to a viewer.
(2) Description of the Related Art
In recent years, online shopping (Electric Commerce, EC) using a computer via Internet has been becoming more popular because of diffusion of broadband high-speed lines such as Asymmetric Digital Subscriber Line (ADSL) and optic fiber. Moreover, in progressing of digital terrestrial television, EC in the shape of uniting an ordinary broadcasting such as a drama image and a shopping is imagined to be realized in digital television, and is anticipated to be established as a killer application in the near future. Such a convenience of online shopping means that it is possible to perform from ordering until receiving the ordered product at home.
However a user is not able to view and confirm the actual product firsthand in online shopping. On this account, even though a note stating “the actual product may appear different from what is shown in the image” is put in many shopping sites, complaints and returned products are still many. As a matter of fact, showing texturally rich image as possible on the user's display has been anticipated.
Now with regard to the size of the display which being used by user is about 17 inches and more for computer and digital television while about 2 to 4 inches of mobile type display for mobile information processing devices such as Personal Digital Assistance (PDA) and cellular phone in above mentioned online shopping. The usage ratio of such a mobile type display is imagined to increase more and more in the future.
It is, therefore, an important subjective to show the product in online shopping in high quality on such a small mobile type display, and various ideas have been carried out.
For example in the case where a viewer holding a mobile display moves three-dimensionally, the display device is able to detect the movement using such as acceleration sensor, and to change the image to be shown actively so as to comprehend the entire relatively big size of image in the conventional image display device.
Here a speed component and a displacement component are calculated by executing temporal integration after detecting the components in vertical direction and horizontal direction for acceleration of the image display device 1701 using the position detecting sensor with two directions by the conventional image display device. Accordingly, moving in two-dimensional direction enables to view a part of a big size of image such as newspaper and map by cutting out, and moving in three-dimensional direction, in other words, moving towards depth direction enables to, for example, comprehend the structure of each floor in a building for architectural purpose, and to inspect a brain scan image perceptively for medical purpose. (for example, refer to Japanese Laid-Open Patent Application No. 2002-7027, page 4, FIG. 2)
Besides in order to comprehend an image of large volume with low-cost and usefully, there is an example of an effect that a user changes a display image for moving display toward the depth of the display, and zooming in or out the display image depending on the moving direction toward deep side or front side. According to this, a viewer is able to view a display of the image display unit as a window like moving around a three-dimensional space. (for example, refer to Japanese Laid-Open Patent Application No. 2000-66802, page 3, FIG. 1)
Additionally there is an image display method that plural numbers of images with different reflection conditions of a static subject by lighting are obtained, and the plural numbers of frame images are shown on the display in turn by switching the image display. In this example, it is described that a camera for viewer's monitoring is equipped in a display, and the display may show the image by switching over to the selected frame image depending on the eye view of viewer seeing the display. Instead of reading image, plural number of original images with different specular reflection condition of a static subject by lighting may be created using CG. (for example, refer to Japanese Laid-Open Patent Application No. 2003-132350, page 16, FIG. 6)
Moreover, there is an example that showing a big area by calculating a depth distance by detecting a viewer's view, operating a window and selecting a menu are performed on a mobile display equipping a small-sized camera. (for example Tomoya Narita, Yu Shibuya, Yoshihiro Tsujino “Display system possible to perform view control by user's view” Human Interface Society report Vol. 3, No. 3, pp. 11 to 14).
However when a user performs online shopping using a mobile display, only easy image processing, that shows a bigger image than the display size, or zooms in or out with the mobile display, with conventional image display device, as feedback operation are available, even the user attempts in different operations. Such image processing is unable to realize a reality like lo holding and viewing an actual product, which being anticipated for online shopping and the like p For example, in the case where a user sees a knitted sweater, it is expected to show a product image of which the user is able to virtually feel the texture of wool.
Surface reflection characteristics that being varied depending on lighting, and a method showing simulated texture of textile weaving has been considering, yet such ways lack ideas including the size of subject and the distance between the user and the subject. It is therefore unable to realize a display of contrasting characteristics such as, for example, a) recognizing the entire image of the subject when viewing from afar, and b) equating the surface of the subject with the display screen and observing the surface texture as if being held directly in the viewer's hand, when viewing at close range.
Accordingly, in order to perform viewing the subject on a display holding by hand, it is not easy by a conventional desktop type display, it is, therefore, obvious that mobile type image display device is preferable. However conventionally it is difficult to show a lot of information on a small display area with the image display device equipping mobile display.
In view of the aforesaid problem, it is an object of the present invention to provide an image display device and an image display method for displaying a product with the highest reality like seeing the product holding by hand for a viewer, even in an application such as online shopping using a small-sized portable display.
In order to achieve the aforesaid object, the image display device according to the present invention for displaying an image of a subject, includes: a distance detecting unit to detect, as a viewing distance, a distance between a viewer and the image display device; an angle detecting unit to detect, as a viewing direction, one of an angle of the image display device as opposed to the angle of the viewer, and a moving direction of the image display device relative to the viewer; and a display switching controlling unit to switch between a first display mode and a second display mode based on the viewing distance, and to display the image of the subject corresponding to one of the first display mode and the second display mode, wherein the first display mode changes a display magnification based on the viewing distance, and displays the image of the subject, and the second display mode displays the images with different directions of normal on the surface of the subject based on the viewing direction.
According to the image display device of the present invention, in a case where inputted images of the subject product is displayed on the small-sized display such as a mobile display, the distance between the display and the viewer, in other words, the viewing distance is obtained by a sensor so as to view from the actual image to the surface fine structure of the subject continuously depending on the distance.
The generating method for the image displayed here is different based on the viewing distance from apart from the subject to close to the subject. In the area where the viewing distance is apart from the subject, the actual image is zoomed out. On the other hand in the area where the viewing distance is close to the subject, the image is switched to a detailed texture image of the subject of which the actual sample is being measured beforehand or a computer graphics image rendered on an arbitrary surface. In the case where the viewing distance is apart from the subject, it is in a state that the subject is viewed using the display frame as a window, on the other hand, in the case where the viewing distance is close to the subject, the display surface is regarded as the subject surface.
In other words, the user is able to comprehend the entire subject viewing the display as a window, and also the user is able to view the subject with high reality like touching it by drawing the subject closer, thereby practicability of online shopping is improved significantly.
The disclosure of Japanese Patent Application No. 2003-367073 filed on Oct. 28, 2003 including specification, drawings and claims is incorporated herein by reference in its entirety.
The disclosure of PCT application No. PCT/JP2004/013487 filed, Sep. 09, 2004, including specification, drawings and claims is incorporated herein by reference in its entirety.
These and other objects, advantages and features of the invention will become apparent from the following description thereof taken in conjunction with the accompanying drawings that illustrate a specific embodiment of the invention. In the Drawings:
FIG.17 is a drawing showing a configuration of an imaging unit of the third embodiment; and
The image display device according to the embodiments of the present invention for displaying an image of a subject, includes: a distance detecting unit to detect, as a viewing distance, a distance between a viewer and the image display device; an angle detecting unit to detect, as a viewing direction, one of an angle of the image display device as opposed to the angle of the viewer, and a moving direction of the image display device relative to the viewer; and a display switching controlling unit to switch between a first display mode and a second display mode based on the viewing distance, and to display the image of the subject corresponding to one of the first display mode and the second display mode, wherein the first display mode changes a display magnification based on the viewing distance, and displays the image of the subject, and the second display mode displays the images with different directions of normal on the surface of the subject based on the viewing direction.
In this way, the user is able to comprehend the entire subject seeing the display as a window. Further, the subject can be seen with high reality like touching it by drawing the subject closer, so that the practicability of online shopping can be improved widely.
Here, the image display device may further preferably include an approaching state detecting unit to calculate a display magnification of the image based on the viewing distance.
In addition, the approaching state detecting unit may calculate the display magnification ratio (a) as one magnification, by assuming that the subject is located at a predetermined distance, in a case where the viewing distance is not greater than the predetermined distance, and (b) as less than one magnification, by assuming that the subject is located beyond the predetermined distance, in a case where the viewing distance is greater than the predetermined distance, and the display switching controlling unit may display the image of the subject in the first display mode in a case where the display magnification calculated by the approaching state detecting unit is less than one magnification, and may display the image of the subject in the second display mode in a case where the display magnification is one magnification.
In this way, in an application such as online shopping using a small-sized display, it is therefore possible to realize a display of contrasting characteristics such as, for example, a) recognizing the entire image of the subject when viewing from afar, and b) equating the surface of the subject with the display screen and viewing the surface texture as if being held directly in the viewer's hand, when the viewer observes at close range.
In addition, in the first display mode, the display switching controlling unit may select an image of a part of the subject based on the display magnification and the viewing direction, and may display the selected image.
In this way, even in the case where the size of the image display area on such as a portable display is small, selecting a part of the subject is easy.
In addition, the part of the subject is one of predetermined areas, and in the first display mode, the display switching controlling unit may select an image in the area out of the plurality of areas based on the viewing direction and the display magnification, and may display the selected image.
In this way, it is possible to reduce the amount of the database for storing a part of the subject.
In addition, the image display device further may include a marker generating unit to display a first marker indicating one of the 1o predetermined areas in the first display mode, and to display a second marker based on the viewing direction and the display magnification, and the display switching controlling unit may select one area out of the plurality of areas depending on a positional relation between the first marker and the second marker.
In this way, it is possible for the viewer to easily confirm the area which can be magnified, and to make the second marker move.
In addition, the display switching controlling unit may display, in the second display mode, a computer graphics image, the computer graphics image is generated by rendering an optical model which describes one of reflection, scattering and transmission of light on the surface of the subject, and the rendering is executed under a fixed lighting to a surface of one of an arbitrary plane and a curved surface.
In this way, in the case where the viewer sees the subject, it is possible to see with high reality like holding by hand directly.
In addition, the image display system according to the embodiments of the present invention is a system which displays on an image display device an image of a subject stored in the server device, and the image display device is characterized by including: a distance detecting unit to detect, as a viewing distance, a distance between a viewer and the image display device; an angle detecting unit to detect, as a viewing direction, one of an angle of said image display device as opposed to the angle of the viewer, and a moving direction of the image display device relative to the viewer; a display switching controlling unit to switch between a first display mode and a second display mode based on the viewing distance, to request an image of the subject corresponding to one of the first display mode and the second display mode, and to display the image of the subject received upon the request, here, the first display mode changes a display magnification based on the viewing distance, and displays the image of the subject, and the second display mode displays the images with different directions of normal on the surface of the subject based on the viewing direction; and a sending/receiving unit to send an image request signal to the server device based on the request, and to receive an image based on the image request signal via a network, the server device to which the network is connected, includes: a subject database which stores the entire image of the subject, and sends the image based on the image request signal; a magnified image database which stores an image of a part of a subject according to a display magnification, and sends the image based on the image request signal; and a tilt variation close-up image database which stores a tilt variation close-up image generated by tilting, to the direction of normal, the image of the part of the subject, and sends the image based on the image request signal.
Further, the present invention can be embodied not only as an image display device and an image display method including as steps for the distinctive characteristics included by the image display device, but also as a program for directing a computer to execute these steps. It is needless to say that the program can be distributed via a storage medium such as CD-ROM and so on or a transmission medium such as the Internet.
Hereinafter embodiments of the present invention will be described using drawings.
An image display device 110 is a mobile display terminal and the like movable freely held by the viewer. The image display device 110 includes a sensor unit 101, a distance detecting unit 102, an angle detecting unit 103, an approaching state detecting unit 104, a display switching controlling unit 105, a marker generating unit 108, an image display unit 109 and a sending/receiving unit 111 as shown in
Here the image display unit 109 is a unit such as liquid crystal display panel for showing images. The sensor unit 101 is a compound sensor system being comprised of an optical image system such as a camera, an ultrasonic wave sensor, an acceleration sensor or the combination of these sensors in order to detect a usage state of the image display device 110. The distance detecting unit 102 detects a distance between an outputting signal from the sensor unit 101 and the viewer. The angle detecting unit 103 detects the angle of the outputting signal from the sensor unit 101 relative to the position of display held by the viewer by placing the viewer as a reference point.
Further the approaching state detecting unit 104 detects a state how the viewer is approaching the image display unit 109. The display switching controlling unit 105 displays an image processed properly depending on the result of the approaching state detecting unit 104 on the image display unit 109. The marker generating unit 108 generates a marker image to display an approaching position on the image for the viewer.
Further the subject database 113 stores the data of the entire image of the product beforehand. The magnified image database 106 stores the data of image beforehand in order to generate a continuous magnified images focusing on a particular part of the product image (image of a part of a product). The tilt variation close-up image database 107 stores the data of the image beforehand in order to generate an image to view by giving a tilt to the approached detailed image. The sending/receiving unit 111 performs sending and receiving the necessary image data to and from the magnified image database 106 and the tilt variation close-up image database 107. The network 112 connects the image display device 110, the magnified image database 106 and the tilt variation close-up image database 107.
Here a signal outputted from the sensor unit 101 is processed at the distance detecting unit 102 and the angle detecting unit 103. Further, the distance between the image display device 110 and the viewer, and the angle at the position of the image display device 110 held by the viewer as opposed to the angle of the viewer as a reference point are detected. It should be noted that regarding the methods to detect the distance and the angle using the optical image system such as camera, the details are described in the above-mentioned reference, “Display system possible to perform view control by user's view”, the description is therefore not included here.
The display switching controlling unit 105, for product image to be changed and the processing, generates an image by interpolating sequentially with actual time, using discrete images stored beforehand in order to display from the entire image to the detailed image of the product as mentioned hereinafter.
The marker image is composed properly and displayed on the image display unit 109 configuring the mobile display. The function of the approaching state detecting unit 104 is described. In
The distances between the virtual subject 201 and the viewer are represented by D1 and D2, and the distances between the image display device 110 and the viewer are represented by Z1 and Zt, and the actual size in vertical direction of the virtual subject 201 is represented by H, the display size at viewing distance Z1 is H1 and the display size at viewing distance Zt is represented by H2.
Here
In this process, the image of the virtual subject 201 is magnified gradually and the display magnification is increased. When viewing distance reaches Zt, the position is the closest position to the virtual subject 201, and it is called in the closest approaching state. The subject distance is represented as D2 this time. When Zt is equal to D2, the positions of the product surface and display surface are the identical, and the actual size of the product is displayed in magnification of 1:1. This time, even though only the part of the subject can be displayed on the mobile display, it is possible to establish a state that is close to a state providing shininess, texture, graininess and so on like touching and viewing the product. The display magnification m is a ratio between the viewing distance and the subject distance, and can be represented in (equation 1). Table 1 shows relations for the display magnification, the viewing distance, and the display size on the display in each approaching process state and the closest approaching state.
m=Z1/D1=H1/H (equation 1)
Assuming that the subject is H=1 (m), D1=10 (m), Z1=40 (cm) and Zt=20 (cm). In the case where the image display device 110 is drawn toward the virtual subject from 40 (cm) to 20 (cm) in viewing distance, the subject distance D1 is shifted to the subject distance D2, actually 10 (m) to 20 (cm), and the display magnification is increased from 4(%) to 100(%), the display size on the image display device 110 is shifted from 4 (cm), which is close to the vertical size, to 1 (m), which is exceeding over the size of the image display device 110. As a result only 4(%) of a part of the subject, that is 4 (cm) out of 1 (m) in vertical direction of the subject, is displayed on the image display device 110.
The operation of the approaching state detecting unit 104 is, for display magnification m, to judge that when m<1 (m is smaller than one), it is in an approaching process state, while when m=1 (m is equal to one), it is in the closest approaching state. This judgment can be executed using the relation of the viewing distance Z, the virtual subject distance D and the size on the display in table 1. Here the particular value of Zt, assumed to be 20 (cm) in the above, is determined tentatively. Since it is not convenient for adjustment function of human's eye if the viewer views the viewing image display device 110 too closely, the display pixel structure on the image display device 110 is unexpectedly viewed. The distance is, therefore, desirable to be a distinct vision of an ordinary distance for seeing display.
Further it is assumed in the embodiment as D2=Zt, that means the subject is located at the same distance with the viewing distance in the closest approaching state. The relation between the subject and the viewing distance can be determined empirically by way of subjective evaluation test in view of human interface, and basically discretional. Accordingly the judgment for the approaching process state and the closest approaching state can use either display magnification, size on display or viewing distance not limited to table 1, and the threshold is also arbitrary.
In the present invention, the method of how to use the result of distance detection and angle detection which are outputted from the sensor and the control method for display image are different depending on in the approaching process state (a first display mode) and in the closest approaching state (a second display mode).
Now in a case where the state is in an approaching process state, the operation of the image display device 110 is described using
First, the viewer views the entire image of the product. If the viewer wants to view the surface of a sweater in detail, the viewer holds the image display device 110 and makes the device 110 approach the sweater targeting the part of the product image. In this state, the angle detecting unit 103 detects the movement as a distance ΔZ on Z axis, a rotation angle pitch Δθ around X axis and a rotation angle yaw ΔΨ around Y axis. In this state when the angle is slight, the equations below are true, and ΔX and ΔY can be calculated. Other than these, the displacement ΔY on Y axis direction and the displacement ΔX in X axis direction can be detected directly. Hereinafter the amounts of movement of the image display device 110 in an approaching process state are represented as ΔX and ΔY.
ΔX≈Z ΔΨ
ΔY≈Z Δθ (equation 2)
As a result of the detection ΔX and ΔY as equation (2), an approaching marker 304 indicating an approaching position desired by the viewer on the image in the display switching controlling unit 105 moves up/down/left/right, and then composed and displayed on the product image. The relation between the moving direction of the approaching marker 304 and the direction of ΔX and ΔY are not specified, and the moving direction of the approaching marker 304 is assumed to move toward counter direction to the direction of ΔX and ΔY and the subject is displayed in this embodiment. For example, in a case where the viewer moves the image display device 110 by holding it towards left side direction, and the approaching position is shifted to slightly right side from the center, so that the approaching marker 304 is shifted to right direction from the previous position on image.
The viewer makes the image display device 110 move along Z axis by holding the device 110 as above, concurrently with adjusting by seeing the approaching marker 304, and the displacement ΔZ of the movement is detected by the distance detecting unit 102.
Subsequently, the display switching controlling unit 105 displays magnified image of the product image by placing the above-mentioned approaching area as a center according to the moved distance of ΔZ in Z axis direction. The display magnification can be set up as a ratio of Z1 and D1 as shown in table 1.
As shown in the above-mentioned embodiment, the above angle is not detected by detecting the line of sight of the viewer, the viewer is not able to move the line of sight freely on the display because the size of the display is small in the image display device 110, so that the line of sight is always fixed on a straight line connecting the viewer's eye and the mobile image display device 110. However as shown in
Accordingly in a case where the viewer moves in XY-dimensional area holding the image display device 110 without changing Z value, the image itself on the display is not shifted, but only the position of the approaching marker 304 is shifted in the approaching process state of the embodiment.
Next an image displayed in the approaching process state described in
At first, three of (A), (B) and (C) are fixed as approachable areas in
As a next step, the viewer holds the image display device 110 and moves to show the approaching marker 304 as shown in
Different types of viewer interface are foreseeable for this section. For example, the target marker 401 is not shown on the product image at the beginning. When the approaching marker 304 lo is appeared by moving the image display device 110, the target marker 401 nearest to the approaching marker 304 may be displayed at first.
Alternatively as shown in
Next in the approaching process state, the approaching area is selected for still image, while the viewing distance approaches the corresponding area, and then the magnified image stored in the magnified image database 106 is displayed as shown in
The magnified image database 106 stores sequential magnified images for the respective approachable areas (A), (B) and (C) beforehand, so that the magnified images can be displayed, or the high-resolution image is processed, so that sequential magnification can be realized. In any case, since the area requiring high resolution image is limited to the approachable area, the amount of accumulated image is comparatively smaller than the accumulated high resolution image of the entire subject advantageously. Besides in this process, the images can be obtained using a device mentioned hereinafter.
Next the closest approaching state is described using
Taking a sweater as an example for product image, the closest approaching state means that the viewer approaches a particular part on the sweater closely enough to examine such as surface pattern, weaving, textile quality and texture, when the viewer sees the actual product. In this case, the display switching controlling unit 105 displays an image by a different operation from an approaching process state.
First, in the closest approaching state, a specified area of product image is displayed on the entire area of the display unit 109, and the knitting stitches of sweater is shown in this example. Here when the viewer holds the image display device 110 and tilts the device 110 like the way mentioned-above, the angle detecting unit 103 detects the tilt of the image display device 110 as a distance ΔZ on Z axis, a rotation angle pitch Δθ around X axis and rotation angle yaw ΔΨ around Y axis.
In the closest approaching state, the distance ΔZ on Z axis direction is subject to be used only at a time of returning to the approaching process state again, in a case where the subject distance D is greater than the viewing distance Zt. In fact the two angles of Δθ and ΔΨ are chiefly detected, and an image in a state that the tilt of the subject surface being varied for the viewer under fixed lighting and fixed eyesight is displayed, the actual texture, graininess and so on of the surface can be shown with high reality.
And now the method of image display in the closest approaching state as mentioned-above is described using the flowchart in
Subsequently, the approaching state detecting unit calculates a display magnification at S603, and then judges the viewing state. Now it is obvious from the (table 1) that this detection may be performed by viewing distance other than the value of the display magnification. Here in the case where the state is detected as the approaching process state, proceeding to S606, and in the case where the state is detected as the closest approaching state, proceeding to S604.
First, in the approaching process state, the marker generating unit 108 shows the approaching marker 304 and the target marker 401 on the image, and then the display switching controlling unit 105 executes selection of approaching area from (ΔΨ, Δθ) to (ΔX, ΔY) at S607.
Subsequently, the display switching controlling unit 105 searches the magnified image database 106 at S608, and outputs sequentially the corresponding magnified image to the image display unit 109 as the image of approaching area to be targeted. And then the image display unit 109 displays the magnified image superimposing on the product image.
Next in the closest approaching state, at S604, the display switching controlling unit 105 calculates the variation of tilt based on angles (ΔΨ, Δθ), searches the tilt variation close-up image database 107, and reproduces the variation of tilt for close-up image.
Next the image display unit 109 displays the searched close-up image on the image display unit 109. Here the variation of the viewing distance Z does not change the close-up image, and when the viewing distance Z is increased, the state returns to approaching process again. Accordingly in a system where the state is changed based on the viewing distance Z, there is a possibility that the state may be fluctuated wildly by noise from a sensor system. In that case, it is effective to take methods giving a sort of hysteresis characteristic and smoothing characteristic to the system.
Hereinbefore, the image display method and the image display device 110 of the first embodiment are described.
Now the utilized image database obtaining method is described.
The tilting stage 702 is a stage to place a sweater as the subject 701, and the tilt is varied. The automatic swivel device 703 has a structure to tilt the tilting stage 702 in two degrees of freedom ΔΨ and Δθ in normal direction of a face, and a structure to move the tilting stage 702 in parallel to XY axes direction. The lighting lamp 704 gives light to the subject 701 under angle of incidence about 22 degrees through the lens 705 creating parallel light.
The camera 706 is set up at a position within a range from afar to close in distance in Z axis direction, which is in vertical direction to XY axes dimension, and taking the subject 701.
In
First, a procedure for preparing an image database of the magnified images of the subject 701 is described.
Here in the approaching process state, it is necessary to create the magnified images which provide sense like approaching serially and gradually to the corresponding part by selecting approaching area to still images.
First, the specified area 711 corresponding to the approachable areas (A), (B) and (C) on the subject 701 are determined, the automatic swivel device 703 is moved in parallel and then the camera 706 is set up to be located on directly above of approachable area.
Next, the camera 708 takes a series of images in the state of approaching the subject by moving to Z axis direction and the images taken are stored in the magnified image database 106.
As mentioned above, a magnified image of approachable area in a state of approaching process can be created.
Next a procedure for preparing an image database of the tilt variation close-up image of the subject 701 is described.
In the closest approaching state, as shown in
Further, the display switching controlling unit 105 creates an actual display image out of two kinds of image database above. In the approaching process state, the magnified image database 106 is searched, if necessary, and a magnified image with interpolating is displayed, while in the closest approaching state, the tilt variation close-up image database 107 is searched referring to Δθ and ΔΨ, and the tilt variation close-up image is interpolated properly when necessary, and displayed.
It should be noted that the relation between the viewing distance and the display magnification in the approaching process state, for example like a direct line 801 shown in
Besides, as for the viewing distance Zmax, the viewing distance Zmin, the viewing distance Zt as the closest approaching state, the values set up beforehand are used, and also the viewer is able to set up as an initial set up. For example the viewing distance Zmax can be set up at a state where the viewer extends his/her arm to the maximum extent holding the image display device 110, the viewing distance Zmin can be set up at the closest position to the image display device 110 where the viewer is able to confirm the image, and the viewing distance Zt can be set up at the position where the viewer desires as the closest approaching state.
In the first embodiment, the image display device 110 using images taken under fixed lighting for taking images as the closest approaching image corresponding to variations in two degrees of freedom for the tilts of normal of the subject surface is described. In the second embodiment, it is described about a case that the viewer uses this image display device inside and outside of building. The structure to display by compounding the closest approaching image which would be viewed under the current lighting, in a case of viewing the closest approaching image is also described.
As a method to describe surface characteristic under a general lighting, in the recent computer graphics field, the conceptions such as the bidirectional reflectance distribution function (BRDF) and the bidirectional texture function (BTF) which is expanded BRDF to two-dimensional image are often used. The BRDF is a function with four variables of (α, β) being expressed the angle of incidence of lighting to only one point on the surface on a spherical coordinate, and (θee, φe) being expressed the view angle on a spherical coordinate, and the function is defined that radiance L is normalized by illuminant E like (equation 3).
BRDF(α, β, θe, φe)=L(θe, φe)/E(α, β) (equation 3)
BTF is also being expanded BRDF to an image area having an arbitrary area, and is a function with six variables (α, β, θe, φe) and (X, Y) in total, BTF may be called as three-dimensional texture to conventional texture. In a case where the illuminant is a constant in the predetermined area, BTF is obtained by normalizing with illuminant from a certain angle of luminance image from view angle direction like (equation 4).
BTF(α, β, θe, φe, X, Y)=L(θe, φe, X, Y)/E(α, β) (equation 4)
In the second embodiment, it is an object to perform display in rich reality further using BTF.
The image display device 905 is such as a mobile display terminal which being held by a viewer and movable freely, and the device 905 includes the lighting direction detecting unit 901, the lighting detecting unit 902, the rendering unit 904, the sensor unit 101, the distance detecting unit 102, the angle detecting unit 103, the approaching status detecting unit 104, the display switching controlling unit 105, the marker generating unit 108, the image display unit 109 and the sending/receiving unit 111. The image display device 905 is connected to the subject database 113, the BTF database 903, the magnified image database 106 and the tilt variation close-up image database 107 via the network 112.
L(θe, φe, X, Y)=BTF(α, β, θe, φe, X, Y)×E(α, β) (equation 5)
The rendering of computer graphics on the basis of image using the observational BTF needs a three-dimensional model of the subject or surface normal, additionally it is necessary to measure using particular equipment such as laser scanner. However the subject is an item whose form is variable like sweater in the embodiment, and BTF is applied to the closest approaching image being relatively local, so that it is not necessary to obtain the surface form of the subject.
Here in the embodiment as shown in
The operation method of the imaging unit 1103 is described. The operation in the approaching process state is not included as it is the same as
Next an angle β is rotated at every predetermined angle, lighting angle a is rotated from 22.5° to 157.5° on the subject rotation structure 1102, and this measurement is repeated. The measurement including two degrees of freedom of lighting angle (α,β) is completed. This result is stored as the BTF database 903.
It should be noted that although the camera 706 is fixed and the lighting angle a is varied in the imaging unit 1103 in
Next, a flowchart for an image display method of the second embodiment is shown in
First, the lighting direction detecting unit 901 performs measurement for the angle of viewing lighting, and the lighting detecting unit 902 performs measurement for illuminant at S1201.
Second, the display switching controlling unit 105 outputs a BTF request signal at S1202 for searching a lighting angle and the BTF corresponding to illuminant to the sending/receiving unit 111, and the sending/receiving unit 111 sends the BTF request signal to the BTF database 903 via a network, and then the BTF corresponding to the BTF request signal is sent to the image display device 905 in the BTF database 903.
Next, the rendering unit 904 generates surface for mapping of the received BTF at S1203, and then the BTF is mapped considering a lighting direction and a normal of the surface, further generates an image rendered as the closest approaching image.
Next the image display 109 displays the generated closest approaching image at S1204.
Additionally, the function used for rendering is the BTF because the surface of sweater where texture being prevailing is exemplified, in a case where the surface is smooth, the same effect can be obtained even the BRDF, and the bidirectional scattering surface reflectance distribution function (BSSRDF) may be used in a case where internal scattering being prevailing such as paper surface.
The image display device 905 in the first and the second embodiment displays only the change of image in viewing to the variations of ΔΨ and Δθ in the closest approaching state. On the other hand, the image display device 905 of the third embodiment to be provided is able to display a magnified image which being a detailed image by reflecting slight variations by the distance of close-up image to variation of a viewing distance by using a database considering concept of distance by extending the BTF.
Next,
The DBTF is a seven-variation texture function to which the conventional BTF (α, β, θe, φe, X, Y) is extended in a distance direction Z, and is defined with the amount of normalization of luminance image from eyesight direction at the position of the viewing distance Z by illuminant of lighting as (equation 6).
DBTF(α, β, θe, φe, X, Y)=L(θe, φe, X, Y, Z)/E(α, β) (equation 6)
In the DBTF database 1301, the DBTF which being obtained by measurement of specified area of the subject performed by particular device beforehand is stored. The rendering unit 904 generates a luminance of detailed image of the subject which would be viewed under a viewing lighting as luminance represented by the product of the BTF and the lighting as (equation 7).
L(θe, φe, X, Y, Z)=DBTF(α, β, θe, φe, X, Y)ΔE(α, β) (equation 7)
In the embodiment, the subject is an item whose form is variable like sweater, and when the image is a detailed comparatively local image, it is not necessary to obtain a subject form due to the DBTF. The virtual plane and the curved surface are assumed to be the virtual curved surface 1002, and the DBTF is applied to the virtual curved surface 1002, so that detailed images rendered can be generated easily and can be displayed on the image display device 1302.
The operation method of the imaging unit 1502 is described. The operation in the approaching process state is not included as it is the same as
Next, the DBTF measurement in a case of the closest approaching state of the third embodiment is described. First, the pixel value viewed by the camera 1501 are directly used as a luminance value, the optoelectric conversion function (OETF) of the camera 1501, therefore, should be calibrated to make the function linear to luminance beforehand.
Next, the camera 1501 is moved to a place at the closest approaching state, the lighting angle a is fixed at an initial position of 22.5° to camera sight angle, and then the subject rotation structure 1102 is fixed to at the position of β=0° to XY coordinates system. Under the above-mentioned setting condition, the tilting stage 703 is tilted with two variables ΔΨ and Δθ, and the images are taken serially again.
Subsequently, the lighting angle a is shifted to the position of 45° to camera sight, and an image is taken by tilting with two variables ΔΨ and Δθ again. This is repeated up to 157.5° at every 22.5° of lighting angle. Next the angle β is rotated at every predetermined angle, the lighting angle α is rotated from 22.5° to 157.5° on the subject rotation structure 1102 again, and this measurement is repeated. The measurement including two degrees of freedom (α, β) of lighting angle is completed.
Next the camera 1501 is shifted closer to the subject than the position of the closest approaching state of the second embodiment, and the measurement is repeated again. Note that in the imaging unit 1502, the subject distance can be varied by moving the camera 1501 in Z axis direction, alternatively, the subject distance also can be varied by using a zooming function of the camera 1501.
The above-mentioned result is stored as the DBTF database 1301. It should be noted that in the imaging unit 1502 of
In
First, the approaching state detecting unit 104 examines a display magnification again at S1601, in a case where the display magnification is m>1, proceeding to S1602, and in a case where m=1, proceeding to S1202. It should be noted that the closest approaching condition is m=1 here, alternatively the condition of the closest approaching state can be set as 0.9<m<1.1, which considers viewer's convenience.
Second, the display switching controlling unit 105 outputs the DBTF request signal for searching the DBTF corresponding to the lighting angle and the illuminant to the sending/receiving unit 111 at S1602, and the sending/receiving unit 111 sends the DBTF request signal to the DBTF database 1301 via a network, and sends the DBTF corresponding to the DBTF request signal to the image display device 1302 in the DBTF database.
Afterward, the display switching controlling unit 105 performs mapping for the DBTF, and generates a detailed image rendered at S1203 like
Additionally, the function used for rendering is the BTF, since the surface like sweater on which texture being prevailing is exemplified. In a case where the surface is smooth, the same effect can be obtained even the BRDF, and the BSSRDF may be used in the case where internal scattering being prevailing such as paper surface.
It should be noted that in the above respective embodiments, it is described about a case that all images are obtained from the respective databases and are displayed, however the other possible case can be conceivable. For example, it is assumed that the viewer takes an image of product at a store initially. When the image of product is displayed afterwards, the actual images taken by the viewer is displayed as the entire image, and the respective images of product in the respective database can be obtained and be displayed for the other zooming-in images.
Although only some exemplary embodiments of this invention have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention.
An image display device and an image display method of the present invention are useful, because, in the case where an inputted image of a subject product is displayed on a small-sized display such as a mobile display in an application such as online shopping, a viewer is able to comprehend the entire subject by viewing the display as a window even on a small display area of the mobile display, further the viewer is able to view the subject product with high reality as if the viewer touched the product with hand by drawing the image display device closer, so that the practicability of online shopping can be improved significantly. Moreover the image display device and the image display method can be used for medical purposes, and also for museums such as digital archives which are effective in a case of combination of actual pictures and computer graphics.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2003-367073 | Oct 2003 | JP | national |
This is a continuation application of PCT application No. PCT/JP2004/013487 filed Sep. 9, 2004, designating the United States of America.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP04/13487 | Sep 2004 | US |
| Child | 11402793 | Apr 2006 | US |
