Patent Grant
6954223
The present invention relates to a stereoscopic image generating apparatus for generating stereoscopic images to be displayed on a n-eye type of stereoscopic image display apparatus, for every frame, and a game apparatus comprising the stereoscopic image generating apparatus.
In recent years, developments in stereoscopic image display apparatuses wherein images are seen so as to stand out from screens thereof have been advanced. The stereoscopic image is achievable by voluntarily generating both eyes parallax caused by an interval between a right eye and a left eye. That is, the stereoscopic image display apparatus provides different images to a right eye and a left eye of a person who watches images, and thereby expresses a stereoscopic sense that is images are viewed so as to stand out. As the method of providing the both eyes parallax, a Lenticular system or a Parallax Barrier system has been known.
In case the stereoscopic display apparatus is a n-eye type, the stereoscopic image to be displayed on the stereoscopic image display apparatus is composed of n-eye images, that is images viewed from n predetermined directions, and thereby generated. The stereoscopic image generation algorithm is a technique which is well known in “3D Image-Processing Algorithms that Take Account of a Lenticular Array's Sampling Effect (3D Image Conference 1996)” or the like, so that the explanation thereof will be omitted.
The above-described stereoscopic image generating algorithm is one of algorithms for generating a stereoscopic image on the basis of a predetermined static image without adhering to the concept of time. Accordingly, it is possible to put the algorithm achievable of the display of dynamic images expressed in the time-continuous display of a plurality of stereoscopic images generated for every frame, that is, frame after frame, at a real time, such as so-called cartoons which are leafed, to practical only by various types of inventions.
Whether the stereoscopic image display apparatus can generate and display the dynamic images at a real time or not depends on the following proposition. That is, it depends on whether the stereoscopic image generating apparatus can generate the stereoscopic images for every frame, that is, frame after frame, time-continuously, or not. More specifically, in case the stereoscopic image generating apparatus has a construction of performing the stereoscopic image generating algorithm only as a software processing, because the frequency of accesses by the stereoscopic image generating apparatus to n-eye original images of each frame becomes high, when the stereoscopic image generating apparatus generates stereoscopic images, there is a possibility it is prevented that the stereoscopic image generating apparatus generates stereoscopic images for every frame, or all n-eye original images are not completed.
Further, because it is necessary that a game apparatus, a three-dimensional CAD system or the like, comprising the stereoscopic image generating apparatus generates not only stereoscopic images but also n-eye original images, it is rigidly restricted on time.
It is an object of the present invention to provide a stereoscopic image generating apparatus for generating stereoscopic images to be displayed on a n-eye type of stereoscopic image display apparatus, for every frame, and displaying dynamic images on the stereoscopic image display apparatus, and a game apparatus comprising the stereoscopic image generating apparatus.
In accordance with a first aspect of the present invention, a stereoscopic image generating apparatus (for example, a stereoscopic image generating apparatus 10 shown in
Herein, the n viewing images means n images of one object viewed from n directions. For example, four viewing images are images generated on the basis of one object viewed from a far left direction, a left direction, a right direction and a far right direction, or the like. Further, the fact the n viewing images are inputted to the input image storage memory for every frame means that n viewing images are collectively inputted to the input image storage memory for every frame time-continuously. Further, each viewing image is stored in a specified storage address of the input image storage memory, and thereby the parallel interleave to the viewing images is realizable. That is, for example, concerning the specified viewing image, in case the viewing image and the stereoscopic image are color images addresses in which image data of the specified sub pixel, that is, color data of the sub pixel are stored, and in case the viewing image and the stereoscopic image are black and white color images addresses in which image data of the specified pixel, that is, color data of the pixel are stored, are always fixed those. Therefore, the sampling when the viewing images are interleaved, can be performed by mechanically reading out image data, that is, color data from addresses as an object of sampling, and the interleaver which is a H/W circuit can easily realize the sampling, and further the interleaving by using a fetch or the like.
According to the stereoscopic image generating apparatus of the above-described first aspect of the present invention, because it is possible to realize storing n viewing images and interleaving n viewing images like an assembly-line operation, it is possible to lower the frequency of access to the memory. Further, because the stereoscopic image generating apparatus has a H/W construction comprising the memory for storing viewing images therein and the interleaver only for interleaving the viewing images, it is possible to realize the higher-speed processing in comparison with the case the S/W interleaves viewing images.
In accordance with a second aspect of the present invention, a stereoscopic image generating apparatus (for example, a stereoscopic image generating apparatus 210 shown in
Herein, the fact the n viewing images are inputted to the interleaver for every frame in serial order means that the n viewing images are shifted from each other in time and inputted to the interleaver for every frame.
According to the stereoscopic image generating apparatus of the above-described second aspect of the present invention, it is possible to generate the stereoscopic image by interleaving the inputted viewing images in order of input to the interleaver even if all n viewing images are not completed. Consequently, the side of generating viewing images (for example, an image generation unit 114 shown in
In accordance with a third aspect of the present invention, a game apparatus (for example, a game apparatus 100 shown in
In accordance with a fourth aspect of the present invention, a game apparatus (for example, a game apparatus 200 shown in
According to the game apparatus of the above-described third or fourth aspect of the present invention, because it is enough that the stereoscopic image generating apparatus generates the stereoscopic image and the game apparatus operates the game and generates the game images, it is possible to distribute and perform the processes in parallel. Further, because the game apparatus supplies game images to the stereoscopic image generating apparatus stably, it is possible that the stereoscopic image generating apparatus generates the stereoscopic image at the stable speed.
The game apparatus may be any one of a portable game machine, a consumer game machine, and an arcade game machine. Further, as the system of the stereoscopic image display apparatus, for example, a Lenticular system or a Parallax Barrier system can be given, and it may be any of them.
Hereinafter, a preferred embodiment of the present invention will be explained with reference to figures, in detail.
Although it will be explained that a stereoscopic image generating apparatus is a four-eye type of Lenticular system of color display apparatus according to the embodiment, the present invention is not limited to the embodiment.
First, an interleaver of the stereoscopic image generating apparatus according to the embodiment will be simply explained with reference to FIG. 1.
As shown in
Further, in
Further, in case the stereoscopic image generating 3 apparatus is a four-eye type, the stereoscopic image generating apparatus interleaves the original images 2-0 to 2-3 for every four pixels continuously, and thereby generates the whole composed image. Therefore, in
Therefore, the far left original image 2-0 is composed of a pixel P00 comprising sub pixels r00, g00 and b00, a pixel P01 comprising sub pixels r01, g01, and b01, a pixel P02 comprising sub pixels r02, g02 and b02, and a pixel P03 comprising sub pixels r03, g03 and b03. As well, the left original image 2-1 is composed of pixels P10 to P13 comprising sub pixels r10, g10 and b10 to r13, g13 and b13, the right original image 2-2 is composed of pixels P20 to P23 comprising sub pixels r20, g20 and b20 to r23, g23 and b23, and the far right original image 2-3 is composed of pixels P30 to P33 comprising sub pixels r30, g30 and b30 to r33, g33 and b33.
Various algorithms are designed as an algorithm for the stereoscopic image generating apparatus of generating the composed image based on a plurality of original images. According to the embodiment, the stereoscopic image generating apparatus may carry out any one of various algorithms. Herein, the direct sampling known as the simplest algorithm will be simply explained with reference to FIG. 2.
As shown in
As described above, when the stereoscopic image generating apparatus generates the composed image, it has been known the brightness of which of sub pixels of the original images corresponds to each sub pixel of the composed images. Accordingly, it is possible that the stereoscopic image generating apparatus of the present invention generates the stereoscopic image with higher speed, by using the above-effect.
Hereinafter, two embodiments to which the stereoscopic image generating apparatus of the present invention is applied will be explained.
First Embodiment
The original image storage unit 20 is a memory for storing original images 90 including a far right original image, a right original image, a left original image and a far left original image viewed from four view points, collectively inputted to the stereoscopic image generating apparatus 10 from the outside for every frame, for example, at intervals of 1/60 second per frame, and composed of a RAM or the like.
Further, addresses at which the far right original image, the right original image, the left original image and the far left original image are stored respectively are previously predetermined in the original image storage unit 20. Therefore, the original images 90 inputted to the stereoscopic image generating apparatus 10 for every frame are restored or stored in storage areas of the original image storage unit 20 corresponding to the original images respectively. Accordingly, for example, in case of the far right original image, besides the area in which the far right original image is stored, the addresses in which data corresponding to sub pixels of the far right original images, more specifically, color data are stored are determined previously.
The interleaver 30 selects, that is, samples the sub pixel corresponding to each composed image sub pixel of the composed image to be displayed as the stereoscopic image among sub pixels of the original images 90 stored in the original image storage unit 20, interleaves in parallel, and thereby generates the stereoscopic image. Herein, the algorithm achievable of the parallel interleave will be indicated as follows. That is, the original images 90 inputted to the stereoscopic image generating apparatus 10 from the outside are stored in the predetermined addresses of the predetermined storage areas of the original image storage unit 20. Therefore, color data of sub pixels of the original images 90 read out by the interleaver 30, more exactly, the original images 90 as an object of sampling are stored in the fixed addresses. For example, image data corresponding to the specified sub pixels of the far right original image 2-3, that is, color data are always stored in the fixed addresses. Accordingly, the interleaver 30 reads out image data, that is, color data from the addresses as an object mechanically, and thereby can perform the sampling when interleaving.
Herein, because the algorithm itself of interleaving including the sampling and so on, that is, the algorithm itself of generating the stereoscopic image has been known well in the above-described “3D Image-Processing Algorithms that Take Account of a Lenticular Array's Sampling Effect (3D Image Conference 1996)” or the like, the explanation thereof will be omitted. Further, the interleaver 30 is composed of a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor) or the like, as a dedicated circuit to interleave.
The display unit 40 is a four-eye Lenticular system of stereoscopic image display apparatus comprising a Lenticular screen consisting of a liquid crystal display or the like. The display unit 40 displays the stereoscopic image generated by the interleaver 30 thereon, and thereby displays the image that can be seen through the Lenticular screen stereoscopically thereon.
According to the above-described configuration, the original images 90 are inputted to the stereoscopic image generating apparatus 10 for every frame, the stereoscopic image generating apparatus 10 stores the inputted original images 90 in the predetermined storage areas of the original image storage unit 20, and the interleaver 30 interleaves the stored original images 90 in parallel. Thereby, the stereoscopic image is generated for every frame.
Next, the game apparatus 100 having the stereoscopic image generating apparatus 10 will be explained with reference to FIG. 4.
The input operating unit 120 is a device for inputting instructions to operate an own character of a game executed by the game apparatus 100, to start and stop the game and so on. The input operating unit 120 has the function achievable in input operating buttons and so on.
The data storage medium 130 stores a game program of executing the game, a program of determining a position of a virtual camera for generating stereoscopic images, and so on, therein. The data storage medium 130 has the function achievable in hardware such as a CD-ROM, a memory, a hard disc or the like.
The CPU 110 mainly comprises a game operation unit 112 and an image generation unit 114. The CPU 110 has the function achievable in hardware such as a CISC (Complex Instruction Set Computer) type or a RISC (Reduced Instruction Set Computer) type of CPU, a DSP, a dedicated IC for reading in images, or the like.
The game operation unit 112 reads out the game program from the data storage medium 130 according to the operation instruction outputted from the input operating unit 120, and constructs a game space by executing the read game program. Further, the game operation unit 112 operates positions of the own character and opponent characters in the constructed game space and a position of the virtual camera in the game space according to the operation instruction outputted from the input operating unit 120, and executes the game. Therefore, the game operation unit 112 outputs various types of coordinate data in the game space to the image generation unit 114. According to the embodiment, the position of the virtual camera when the game operation unit 112 operates the position of the virtual camera in the game space is locations of four viewpoints. However, the game operation unit 112 may determine only the position of the virtual camera at one viewpoint, and provide a virtual camera at another viewpoint of the position moved a predetermined distance in the horizontal direction or the vertical direction from the position of the virtual camera at the one viewpoint determined in the game space.
The image generation unit 114 generates images according to the positions of the virtual cameras corresponding to four viewpoints when the various types of coordinate data in the game space are inputted from the game operation unit 112. Therefore, when the image generation unit 114 outputs the generated images as the original images 90 to the stereoscopic image generating apparatus 10, the original images 90 are stored in the original image storage unit 20 of the stereoscopic image generating apparatus 10.
According to the stereoscopic image generating apparatus 10, when the interleaver 30 performs the above-described processing on the basis of the original images 90 stored in the original image storage unit 20, generates the stereoscopic image, and thereby outputs the stereoscopic image to the display unit 40, the display unit 40 displays the stereoscopic image thereon. Herein, the processing of generating the original images 90 is a processing performed by the CPU 110, and the processing of generating the stereoscopic image on the generated original images 90 is a processing performed by the stereoscopic image generating apparatus 10. Accordingly, because it is enough that the CPU 110 performs until the processing of generating the original images 90, the CPU 110 is released from the processing of generating the stereoscopic image, and it is unnecessary that the CPU 110 accesses the memory storing the original images 90 therein. On the other hand, according to the stereoscopic image generating apparatus 10, because the original images 90 are generated for every frame by the CPU 110, the timing when all original images 90 corresponding to four viewpoints respectively are inputted become stable, and it is possible that the interleaver 30 generates the stereoscopic image stably.
Second Embodiment
First, the stereoscopic image generating apparatus 210 incorporated in the game apparatus 200 will be explained.
The stereoscopic image generating apparatus 210 is an apparatus realizable of generating the stereoscopic image in case the original images 90 at the viewpoints are inputted serially, that is, in series, that is, in case all the original images 90 are not completed at the same time. The stereoscopic image generating apparatus 210 comprises a frame buffer 220 and an interleaver 230.
The frame buffer 220 is a memory storing one frame of stereoscopic image therein, and the image completed as a stereoscopic image is displayed on the display unit 40. Herein, the “complete” image is used in contrast with the “incomplete” image, because there is a time to store the “incomplete” image in the frame buffer 220. Hereinafter, in order to make the explanation simple, the “composed image” stored in the frame buffer 220 and “completed” will be called the “stereoscopic image”.
The interleaver 230, performs a series of processing of interleaving the original images inputted therein and the composed images stored in the frame buffer 220, and storing the result in the frame buffer 220, in order, continuously, and thereby generates the stereoscopic images. More specifically, the interleaver 230 performs a processing shown in
That is, as shown in
On the other hand, at the Step S2, when the interleaver 230 determines that the inputted original image is not the first in the frame (Step S2; N), the interleaver 230 reads out the image stored in the frame buffer 220 (Step S4), interleaves the read out image with the inputted original image (Step S5), and restores the interleaved composed image in the frame buffer 220 (Step S6).
After the processing at the Step S3 or the Step S6, the interleaver 230 determines whether one frame of original images, that is, four original images have been processed or not (Step S7). While the interleaver 230 determines that one frame of original images have not been processed (Step S7; N), the interleaver 230 carries out the processing from the Step Si to the Step S7 continuously. When the interleaver 230 determines that one frame of original images have been processed (Step S7; Y), the interleaver 230 ends the present processing.
According to the above-described processing, the stereoscopic image is stored in the frame buffer 220 finally.
Herein, although the interleave performed by the interleaver 230 at the Step S5, is obvious to persons having ordinary skills in the art in the above-described interleave algorithm, it will be again explained simply just to make sure.
Accordingly, in each original image, the sub pixel used as the stereoscopic image is determined previously. That is, at the Step S5 in
Next, the game apparatus 200 incorporating the above-described stereoscopic image generating apparatus according to the second embodiment will be explained. Although the CPU 110, the input operating unit 120, the data storage unit 130 and the display unit 40 constituting the game apparatus 200 are the almost same as the corresponding units of the game apparatus 100 according to the first embodiment, the processing performed by the CPU 110 is a little different from the processing according to the first embodiment.
That is, because it is unnecessary that the image generation unit 114 of the CPU 110 generates the far right original image, the right original image, the left original image, and the far left original image, at the same time, the image generation unit 114 outputs the original images to the stereoscopic image generating apparatus 210 in generating order. Then, the stereoscopic image generating apparatus 210 interleaves in order on the basis of the inputted original images, and outputs the composed image stored in the frame buffer 220, that is, the stereoscopic image to the display unit 40, when interleaving with all original images of one frame.
Therefore, in the game apparatus 200, as well as the case according to the first embodiment, the processing of generating the original images and the processing of generating the stereoscopic image can be distributed, and further, according to the second embodiment, the parallel processing of generating images, such as a pipeline processing can be carried out. That is, in
In other words, because the interleaver 230 performs the processing of generating the composed image first on the basis of the inputted original images without waiting for all original images outputted from the image generation unit 114 to be complete, it is possible to reduce the time the interleaver 230 waits to perform the processing and to shorten the time of generating the stereoscopic image.
According to the second embodiment, it is unnecessary to say that the stereoscopic image generating apparatus 210 does not need a memory having a capacity storing all original images therein.
Although the first embodiment and the second embodiment to which the present invention is applied have been explained, it should also be understood that the present invention is not limited to the above-described embodiment. For example, the game apparatuses 100 and 200 may be applied to any one of a consumer game machine, a portable game machine and an arcade game machine, or, for example, a three-dimensional CAD (Computer Aided Design) system which can be applied to a stereoscopic image generating apparatus or the like of a walk-through model.
Further, as the display unit 40, the present invention can be applied not only to a multi-eye stereoscopic system and a compound-eye stereoscopic system besides a four-eye stereoscopic system but also to another stereoscopic system which needs to interleave when generating a stereoscopic image, for example, a parallax barrier stereoscopic system.
Further, although it has been explained that a color display apparatus is used as the display unit 40 according to the above-described embodiment, a black and white display apparatus may be used as the display unit 40. In the case, the interleaver interleaves not for every sub pixel but for every pixel.
According to the present invention, in the stereoscopic image generating apparatus, it is possible to generate the stereoscopic image for every frame at a real time, and to realize the display of the stereoscopic image as a dynamic image. That is, because it is possible to realize storing original images and interleaving the original images like an assembly-line operation, it is possible to lower the frequency of access to the memory. Further, it is possible to generate the stereoscopic image by interleaving the original images in order of input even if all plurality of original images are not completed. Further, according to the game apparatus incorporating the stereoscopic image generating apparatus therein, because it is enough that the stereoscopic image generating apparatus generates the stereoscopic image and the game apparatus executes the game and generates the game image, it is possible to distribute and perform the processing in parallel. Consequently, it is possible to perform the processing of the stereoscopic image with higher speed, and to reduce the processing time thereof. Accordingly, the present invention is suited for a stereoscopic image generating apparatus for generating stereoscopic images to be displayed on a n-eye type of stereoscopic image display apparatus, for every frame, and displaying dynamic images on the stereoscopic image display apparatus, and a game apparatus comprising the stereoscopic image generating apparatus.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2000-256049 | Aug 2000 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP01/07026 | 8/15/2001 | WO | 00 | 1/24/2002 |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5614941 | Hines | Mar 1997 | A |
| 6061083 | Aritake et al. | May 2000 | A |
| 6064424 | van Berkel et al. | May 2000 | A |
| 6069650 | Battersby | May 2000 | A |
| 6084978 | Taylor et al. | Jul 2000 | A |
| 6573928 | Jones et al. | Jun 2003 | B1 |
| 6801243 | Van Berkel | Oct 2004 | B1 |
| Number | Date | Country |
|---|---|---|
| A 11-27607 | Jan 1999 | JP |
| 11-027607 | Jan 1999 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20020105576 A1 | Aug 2002 | US |
