Patent Grant
8068129
The disclosed technique relates to endoscopes, microscopes and boroscopes, in general and to stereoscopic image pick up devices with color imaging capability, in particular.
Stereoscopic image detection devices are known in the art. Such devices are required to obtain and provide a combination of small cross section and high image quality. It will be appreciated by those skilled in the art that high image quality, in general, is characterized by stereoscopic vision accuracy, color capabilities, high resolution and illumination requirements.
It is noted that conventional methods, which provide stereoscopic images, require a wider optical path than a monocular one. Such a widened optical path enlarges the cross-section required for the detection device considerably. Hence, the requirement for a small cross section is not maintained.
U.S. Pat. No. 5,527,263 to Zobel, et al., is directed to a dual optical path stereo endoscope with simple optical adjustment. U.S. Pat. No. 5,776,049 to Takahashi, is directed to a “Stereo Endoscope and Stereo Endoscope Imaging Apparatus” and provides a device which utilizes a combination of two optical paths with two charge coupled devices (CCD's), capable of variable zoom.
Auto-stereoscopic devices, which utilize one optical system to provide a stereo effect, are also known in the art. Such a device is provided in U.S. Pat. No. 5,603,687 to Hori et al., which is directed to a device with two parallel optical axes and two CCD units. Hori selected an asymmetrical approach, wherein one optical channel has a large aperture for light and details, and the other optical channel provides a parallax image for stereoscopic imagery to the proximal CCD.
U.S. Pat. No. 5,613,936 to Czarnek et al., is directed to a stereoscopic endoscope device which utilizes light polarization and time multiplexing, in order to transmit each different polarized image corresponding to left and right images multiplexed in time, through one optical channel that transfers images from the lateral side of the endoscope shaft. This endoscope has to be inserted deeper into the human cavity to receive a stereo image. It must also be used with a head mounted display device called “switched shutter glasses” that causes eye irritation. It is noted that according to Czarnek each image is received in 25% of the original quality. As much as 50% of the light received from the object, is lost due to polarization considerations and as much as 50% of the remaining information is lost due to channel switching.
U.S. Pat. No. 5,588,948, to Takahashi, et al., is directed to a stereoscopic endoscope. The stereo effect is produced by having a dividing pupil shutter, which splits the optical path onto the left and right sides, and the up and down sides. These sides are alternately projected on a proximal image pick up device, using time multiplexing. According to another aspect of this reference, a distal CCD is included, which is divided to left and right sides with a shading member separating them, for achieving space multiplexing.
U.S. Pat. No. 5,743,847 to Nakamura et al., is directed to a “Stereoscopic Endoscope Having Image Transmitting Optical-System and Pupil Dividing Unit that are Axially Movable With Respect to Each Other”, which uses a plural pupil dividing means and one optical channel. U.S. Pat. No. 5,751,341 to Chaleki et al., is directed to a “Stereoscopic Endoscope System”, which is basically a two channel endoscope, with one or two proximal image sensors. A rigid sheath with an angled distal tip could be attached to its edge and be rotated, for full view.
U.S. Pat. No. 5,800,341 to Mckenna et al., is directed to an “Electronically Steerable Endoscope”, which provides different fields of view, without having to move the endoscope, using a plurality of CCD cells and processing means. U.S. Pat. No. 5,825,534 to Strahle, is directed to a “Stereo Endoscope having a Folded Sight Line” including a stereo-endoscope optical channel, having a sight line folded relative to tube axis.
U.S. Pat. No. 5,828,487 to Greening et al., is directed to a “Stereoscopic Viewing System Using a Two Dimensional Lens System” which in general, provides an alternative R-L switching system. This system uses a laterally moving opaque leaf, between the endoscope and the camera, thus using one imaging system. U.S. Pat. No. 5,594,497 to Ahern, describes a distal color CCD, for monocular view in an elongated tube.
The above descriptions provide examples of auto-stereoscopic disclosed techniques, using different switching techniques (Time division multiplexing) and polarization of channels or pupil divisions (spatial multiplexing), all in an elongated shaft. When color image pick up devices are used within these systems, the system suffers from reduced resolution, loss of time related information or a widened cross section.
The issue of color imagery or the issue of a shaft-less endoscope is not embedded into any solution. To offer higher flexibility and to reduce mechanical and optical constraints it is desired to advance the image pick-up device to the frontal part of the endoscope. This allows much higher articulation and lends itself easily to a flexible endoscope. Having a frontal pick up device compromises the resolution of the color device due to size constraints (at this time).
U.S. Pat. No. 5,076,687 to Adelson, is directed to an “Optical Ranging Apparatus” which is, in general a depth measuring device utilizing a lenticular lens and a cluster of pixels.
U.S. Pat. No. 5,760,827 to Faris, is directed to “Pixel Data Processing System and Method for Producing Spectrally-Multiplexed Images of Three-Dimensional Imagery for Use in Stereoscopic Viewing Thereof” and demonstrates the use of multiplexing in color and as such, offers a solution for having a color stereo imagery with one sensor. Nevertheless, such a system requires several sequential passes to be acquired from the object, for creating a stereo color image.
U.S. Pat. No. 5,812,187 to Watanabe, is directed to an Electronic Endoscope Apparatus. This device provides a multi-color image using a monochromatic detector and a mechanical multi-wavelength-illuminating device. The monochromatic detector detects an image, each time the multi-wavelength-illuminating device produces light at a different wavelength.
U.S. Pat. No. 6,306,082 B1 issued to Takahashi, et al., and entitled “Stereoendoscope wherein images having passed through plural incident pupils are transmitted by common relay optical systems”, is directed to an apparatus, namely, an endoscope wherein images, having passed through plural incident pupils, are transmitted by a common relay system, and reconstructed at an observation point to provide a stereoscopic image. According to the reference, illuminating light is transmitted by a light guide. Light reflected from the illuminated objects passes through non-superimposed pupils and transmitted to the rear side by a common relay system having a single optical axis. The transmitted images are formed on separate image taking surfaces to allow for a stereoscopic image to be formed.
U.S. Pat. No. 5,121,452 issued to Stowe, et al., and entitled “Fiber Optic Power Splitter”, is directed to a method for manufacturing fiber optic power splitters. The fiber optic power splitter is a unitary, single-mode fiber, fused structure which is composed of four, up to seventeen or more fibers, which provide uniform splitting of input optical power among the fibers. The fiber optic power splitter includes a central fiber and identical surrounding fibers, which are sized prior to fusion, such that mutual contact is achieved. In this manner, each of the surrounding fibers touches the central fiber and the neighboring fibers. In this construction, the surrounding fibers are of the same diameter and the central fiber has a different diameter. Optical power input in the central fiber distributes among the surrounding fibers. The optical power output in the central fiber and the surrounding fibers is monitored during the fusion process, and the fusion process is stopped when the desired fraction of the optical power appears in a surrounding fiber.
In Handbook of Optics, Volume 2, McGraw-Hill, Inc., 1995, p. 15-24, Norman Goldberg discusses the concept of stereo cameras. The structure of a stereo camera is based on the parallax difference between the views of the right and the left eyes. The two lenses in the classic stereo camera are spaced about 65 mm apart, in order to form two images of the subject. Another type of stereo camera uses a reflection system of four mirrors or an equivalent prism system, placed in front of the lens of a normal camera, thereby forming two images of the subject (FIG. 15 on p. 15-25 of the Handbook). According to another method, the subject is required to remain stationary while two separate exposures are made and the camera is shifted 65 mm between the two exposures. This method is employed in aerial stereo photography in which two views are made of the ground, the views being made so many seconds apart.
According to another method, the right and left views of the subject are restricted to the respective eye of the viewer, where the right and the left views are polarized at 90 degrees to one another. The viewer wears glasses with polarizing filters oriented such that each eye sees the view intended for it. In a parallax stereogram, the right and left images are sliced into narrow, interlaced right and left strips. The viewer perceives a three-dimensional view of the subject, while viewing the image through a series of vertical lenticular prisms with a matching pitch.
U.S. Pat. No. 5,233,416 issued to Inoue and entitled “Electronic Endoscope System”, is directed to a system which enables the use of an endoscope having either a normal sensitivity or a high sensitivity solid-state image sensor element. The system includes a rotary color wheel, a light source, a condenser lens, the solid-state image sensor element, such as charge coupled device (CCD), an input switch, a first video processor, a second video processor, an output switch, an analog to digital (A/D) converter, a plurality of storage portions, three digital to analog (D/A) converters, an encoder, a first control means, a second control means, a decoder, a master clock and a CCD drive.
The CCD drive is coupled with the CCD, the first control means, and to the master clock. The first control means is coupled with the input switch, the first video processor, the second video processor, the output switch, the A/D converter, the storage portions, the decoder and to the master clock. The CCD is coupled with the decoder and to the input switch. The input switch is coupled with the first video processor and to the second video processor. The output switch is coupled with the first video processor, the second video processor and to the A/D converter. The storage portions are coupled with the A/D converter, to the three D/A converters and to the second control means. The second control means is coupled with the decoder, the master clock, the D/A converters and to the encoder. The three D/A converters are coupled with the encoder.
The condenser lens is located between the light source and the rotary color wheel. The rotary color wheel is located between the condenser lens and a light guide of the endoscope. The rotary color wheel is provided with three filter zones (red, green and blue). The three filter zones are separated by three color-shifting light-blocking zones. Each filter zone is bisected into uniform halves, by an intermediate light-blocking zone.
The input switch switches the system to the first video processor when the normal sensitivity CCD is employed and to the second video processor, when the high sensitivity CCD is employed. The first control means controls the read-out of the signal charges from the CCD and the second control means controls the display of the images. Each of the first control means and the second control means can operate either in a normal sensitivity mode or a high sensitivity mode. The CCD drive produces pulse signals for the CCD, according to the clock signals of the master clock.
The rotary color wheel provides an image to the CCD in red, green and blue, in sequence. When a normal sensitivity CCD is employed, the system switches to the first video processor, and the first control means, the second control means and the CCD drive switch to the normal sensitivity mode. In this mode, the CCD drive enables the read-out of signal charges from the CCD, between every two color-shifting light-blocking zones. The first controller shifts the resulting image to the storage portions, during each color-shifting light-blocking zone. The second controller constructs a color image for each pulse signal, by combining the three images in red, green and blue which are read-out between every two color-shifting light-blocking zones.
When a high sensitivity CCD is employed, the system switches to the second video processor, and the first control means, the second control means and the CCD drive switch to the high sensitivity mode. In this mode, the CCD drive enables the read-out of signal charges from the CCD, between every two color-shifting light-blocking zones, as well as between every two intermediate light-blocking zones. The first controller shifts the resulting image to the storage portions, during each color-shifting light-blocking zone, as well as during each intermediate light-blocking zone. The second controller constructs a color image for each pulse signal, by combining the three images in red, green and blue which are read-out between every two color-shifting light-blocking zones, as well as between every two intermediate light-blocking zones.
It is an object of the disclosed technique to provide a novel system for stereoscopic imaging, by employing an image receiving assembly whose inlets are spaced apart, and a novel method for operating the same, which overcomes the disadvantages of the prior art.
In accordance with one aspect of the disclosed technique, there is thus provided a stereoscopic device which includes an image directing assembly, an image differentiator and an image detector. The image directing assembly includes a first light inlet for receiving a first image and a second light inlet for receiving a second image, wherein the first light inlet and the second light inlet are spaced apart. The image differentiator differentiates between the first image and the second image and the image directing assembly directs the first image and the second image to the image detector via a common path.
A controller coupled with the image detector and to an image processor, enables the image detector to detect the first image and the second image according to the state of the image differentiator. The image processor produces a stereoscopic image, by processing the detected first image and second image.
In accordance with another aspect of the disclosed technique, there is thus provided a method for producing a stereoscopic image. The method includes the procedures of receiving images of different sides of an object through two spaced apart apertures, directing the images to a common path and differentiating between the images. The method further includes the procedures of detecting the images, processing the detected images and displaying a stereoscopic image according to the processed images.
In accordance with a further aspect of the disclosed technique, there is thus provided a stereoscopic device including a first light filter, a second light filter, a sequential wavelength differentiator, an image detector and an optical assembly located in front of the image detector. The first light filter admits light at a plurality of first ranges of filter wavelengths and the second light filter admits light at a plurality of second ranges of filter wavelengths. The sequential wavelength differentiator is associated with a first set of differentiating wavelengths and with a second set of differentiating wavelengths.
The image detector receives images from the first light filter and from the second light filter. The first set of differentiating wavelengths is included in at least one of the first ranges of filter wavelengths and excluded from the second ranges of filter wavelengths. The second set of differentiating wavelengths is included in at least one of the second ranges of filter wavelengths and excluded from the first ranges of filter wavelengths. A controller is coupled with the image detector, to the image processor and to the sequential wavelength differentiator. The controller enables the image detector to detect the first image and the second image according to the state of the sequential wavelength differentiator. The image processor produces a stereoscopic image, by processing the detected first image and second image.
The sequential wavelength differentiator can be a sequential illuminator, sequentially emitting light at least a portion of the first set of differentiating wavelengths and at least a portion of the second set of differentiating wavelengths. Alternatively, the sequential wavelength differentiator can be a filtering differentiator, differentiating between at least a portion of the first ranges of filter wavelengths and at least a portion of the second ranges of filter wavelengths.
Further alternatively, the filtering differentiator can be a multi-wavelength rotating disk located in front of the image detector, wherein the multi-wavelength rotating disk includes a plurality of filtering sectors. Each of the filtering sectors admits light at different wavelengths selected from one of the first set of differentiating wavelengths and the second set of differentiating wavelengths. The multi-wavelength rotating disk sequentially filters light at the common path and the controller enables the image detector to detect images, according to the angular position of the multi-wavelength rotating disk.
In accordance with another aspect of the disclosed technique, there is thus provided a method for detecting a first image and a second image. The method includes the procedure of determining a plurality of first ranges of filter wavelengths for a first pupil and a plurality of second ranges of filter wavelengths for a second pupil. The method further includes the procedure of sequentially differentiating between a first set of differentiating wavelengths and a second set of differentiating wavelengths. The method includes still further, the procedure of detecting the first image when the first set of differentiating wavelengths is present, and detecting the second image when the second set of differentiating wavelengths is present. The first set of differentiating wavelengths is included in the first ranges of filter wavelengths and excluded from the second ranges of filter wavelengths. The second set of differentiating wavelengths is included in the second ranges of filter wavelengths and excluded from the first ranges of filter wavelengths.
The disclosed technique will be understood and appreciated more fully from the following detailed description taken in conjunction with the drawings in which:
The disclosed technique overcomes the disadvantages of the prior art by providing a continuous imaging stereoscopic apparatus, using a generally lenticular lens layer, a light sensor array and an image processing system.
Reference is now made to
Light sensor array 102 includes a plurality of sensors 110, 111, 112, 113, 114, 115, 116, 117, 118 and 119. Lenticular lens layer 104 includes a plurality of lenticular elements 130, 132, 134, 136 and 138. Each one of the lenticular elements is located above two light sensors, in a way that lenticular element 130 is located above sensors 110 and 111, lenticular element 132 is located above sensors 112 and 113, lenticular element 134 is located above sensors 114 and 115, lenticular element 136 is located above sensors 116 and 117 and lenticular element 138 is located above sensors 118 and 119.
The light sensors 110, 111, 112, 113, 114, 115, 116, 117, 118, and 119, detect light as directed by the lenticular lens elements 130, 132, 134, 136 and 138, and provide respective information to the processor 106. The processor 106 processes this information, produces a pair of images, as will be explained in detail herein below, and provides them to the display units 108R and 108L, which in turn produce visual representations of these images.
In general, each lenticular element directs light rays, which arrive from a predetermined direction to a predetermined location, and light rays which arrive from another predetermined direction, to another predetermined location. Hence, the disclosed technique, utilizes the lenticular lens layer to distinguish between a right view image and a left view image, as is described herein below.
Each of the display units 108R and 108L includes a plurality of display units also known as pixels. Display unit 108L includes pixels 142A, 142B, 142C, 142D and 142E. Display unit 108R includes pixels 144A, 144B, 144C, 144D and 144E. Using these pixels each of the display units 108R and 108L produces an image, according to data provided from the processor 106. The two images, each viewed by a different eye of the user, produce a sensation of a three-dimensional image.
Light rays 124A, and 126A represent a right-side image of the three-dimensional object 150. Light rays 120A, and 122A represent a left side image of the three-dimensional object 150. The optical assembly 152 redirects light rays 120A, 122A, 124A and 126A so as to focus them on a plain which is determined by the light sensor array 102, as light rays 120B, 122B, 124B and 126B, respectively. Hence, light rays 122B and 126B represent a focused right side view of the three-dimensional object 150, and light rays 120B and 124B represent a focused left side view of the three-dimensional object 150.
The lenticular lens layer 104 directs the focused right side view light rays 122B and 126B to light sensors 110 and 118, respectively, as respective light rays 122C and 126C. In addition, the lenticular lens layer 104 directs the focused left side view light rays 120B and 124B to light sensors 111 and 119, respectively. In general, light sensors 111, 113, 115, 117 and 119 detect light rays which relate to a left side view image of object 150, and light sensors 110, 112, 114, 116, and 118, detect light rays which relate to a right side view image of object 150.
Hence, light sensors 110, 112, 114, 116 and 118 detect the right side image of object 150, while light sensors 111, 113, 115, 117 and 119 detect the left side image of object 150. The light sensor array 102 provides data relating to the detected light intensity at each of the light sensors to the processor 106. It is noted that in the following description, the term processor, refers to a control unit which is adapted for a given situation such as a CPU, a controller, a processor, a gated element, a timing unit such as a clock, and the like. Accordingly, the terms CPU, controller, processor, gated element, timing unit, clock, and the like, are interchangeable, with respect to a given architecture or a given method.
The processor 106 processes this data, produces a right side image from the data relating to the right side view and a left side image from the data relating to the left side view, and provides the respective image to the respective display unit 108R and 108L. In the present example, the processor 106 utilizes the data received from sensors 110, 112, 114, 116 and 118 to determine the data provided to pixels 144A, 144B, 144C, 144D and 144E, respectively. Similarly, the processor 106 utilizes the data received from sensors 111, 113, 115, 117 and 119 to determine the data which is to be provided to pixels 142A, 142B, 142C, 142D and 142E, respectively.
According to the disclosed technique, the right side image and the left side image are detected at the same time and hence, can also be displayed at the same time. According to another aspect of the disclosed technique, each of the light sensors 110, 111, 112, 113, 114, 115, 116, 117, 118, and 119, includes a plurality of color sensing elements, which together cover a predetermined spectrum, as will be described in detail herein below.
Reference is now made to
The sensor assembly 202 includes a focusing element, which in the present example is a lens 226, a lenticular lens layer 222, a light sensor array 220, an interface 228 and a light projecting means 224. The lenticular lens layer 222 is attached to the light sensor array 220. According to the disclosed technique, the light sensor array 220 can be any type of sensing array, such as a CCD detector, a CMOS detector, and the like. The light sensor array 220 is coupled with the interface 228, which can also acts as a supporting base.
The stereoscopic display 214 includes two display units, a left display unit 216L (for placing in front of the left eye of the user) and a right display unit 216R (for placing in front of the right eye of the user). Hence, the stereoscopic display 214 is capable of displaying stereoscopic images continuously. Such a stereoscopic display unit is for example the ProView 50 ST head-mounted display, manufactured and sold by Kaiser Electro-Optics Inc., a US registered company, located in Carlsbad, Calif. Another example for a stereoscopic display unit is the virtual retinal display (VRD) unit, which is provided by MICROVISION Inc., a US registered company, located in Seattle, Wash. It is noted that any method, which is known in the art for displaying stereoscopic, and for that matter three-dimensional images, is applicable for the disclosed technique.
The image received from a three-dimensional object is received at the sensor assembly 202, focused by lens 226, optically processed by the lenticular lens layer 222 and finally detected by the light sensor array 220. The lenticular lens layer 222 directs light coming from one predetermined direction to predetermined light sensors of the light sensor array 220, and light coming from another predetermined direction to other predetermined light sensors of the light sensor array 220. Accordingly, light sensor array 220 detects two images of the same object, a right side image and a left side image, each from a different direction. This aspect of the disclosed technique is described in detail hereinabove, in conjunction with
An electronic representation of this information is partially processed by the interface 228 and then provided to the interface 210, via flexible cord 218. It is noted that flexible cord 218 may include digital communication linking means such as optic fibers or electrical wires, for transferring data received from light sensor array 220, as well as light guiding conducting means for conducting light from light source 206 to the light projecting means 224. According to the disclosed technique, flexible cord 218 can be replaced with a rigid cord (not shown), if necessary.
The data received at interface 210 includes information, which relates to the two images and has to be processed so as to distinguish them from each other. As the processor 208 processes the information, it uses the memory unit 204 as temporarily storage.
After processing the information, the processor 208 produces two matrices each being a reconstructed representation relating to one of the originally detected images. The processor provides these matrixes to the stereoscopic video generator 212, which in turn produces two respective video signals, one for the left view image and another for the right view image.
The stereoscopic video generator 212 provides the video signals to the stereoscopic display 214, which in turn produces two images, one using right display unit 216R and another using left display unit 216L.
It is noted that the general size of the sensor assembly 202 is dictated by the size of the sensor array and can be in the order of a few millimeters or a few centimeters. This depends on the size of each of the sensors in the array and the total number of sensors (i.e. the required optical resolution).
According to one aspect of the disclosed technique, each of the sensors in light sensor array 220, is a full range sensor, which yields data relating to a gray scale stereoscopic image. According to another aspect of the disclosed technique, each of the sensors in the light sensor array, can be adapted so as to provide full color detection capabilities.
Reference is now made to
Reference is further made to
The data provided by these sensors can later be utilized to reconstruct an image in full color. Similarly, the lenticular element 318 directs light, which arrives from the left (left view image), to the right section of the super-pixel 300, where it is detected in full spectrum by sensors 308, 310 and 312.
Reference is now made to
Lenticular element 334(1) covers the first column of super-pixels 340 from super-pixel 340(1,1) to super-pixel 340(1,N). Lenticular element 334(2) covers the second column of super-pixels 340 from super-pixel 340(2,1) to super-pixel 340(2,N). Lenticular element 334(3) covers the third column of super-pixels 340 from super-pixel 340(3,1) to super-pixel 340(3,N). Accordingly, each of the lenticular elements of the lenticular lens layer covers an entire column of super-pixels.
It is noted that a super-pixel according to the disclosed technique can include sensors in any set of colors such as red-green-blue (RGB), cyan-yellow-magenta-green (CYMG), infra-red, ultra-violet, and the like, in any arrangement or scheme such as column, diagonals, and the like. It is noted that such a set of colors can be achieved either by using specific color sensitive detectors or by using color filters over the wide spectrum detectors.
The output of a conventional CYMG sensor array can include a plurality of values, each of which is equal to the sum of two cells in the same column and in adjacent rows. The following sums may apply in a conventional CYMG sensor array—Cyan+Magenta, Yellow+Green, Cyan+Green and Yellow+Magenta.
Reference is further made to
Reference is further made to
The color arrangement which is provided for super-pixel 370 is typical for vertical light detection arrays, where each column of sensors is coated with light filtering layer of a different color. As can be seen in
Reference is now made to
As can be seen from
Reference is now made to
In step 402, the apparatus detects each light sector separately, using a plurality of light detectors, each detecting a portion of its respective sector. With reference to
In step 404, the apparatus 200 determines the light characteristics as detected by each of the light sensors, at each of the sub-sectors. In step 408, the apparatus 200 utilizes the data, which was accumulated from selected sub-sectors to determine and produce an image representing a view from one side. In step 406, the apparatus 200 utilizes the data, which was accumulated from other selected sub-sectors to determine and produce an image representing a view from another side. In step 410, the apparatus 200 displays both images using a continuous stereoscopic display device.
According to a further aspect of the disclosed technique, information from selected pixels can be used to enhance information for other pixels. For example, color information of pixels, which are associated with a first color, is used for extrapolating that color at the location of another pixel, associated with a second color.
Reference is further made to
In step 420, the system, according to the disclosed technique, selects a pixel sensor, associated with a first color. With reference to
In step 422, the system determines pixels, associated with a second color, in the vicinity of the selected pixel. It is noted that these pixels can also be restricted to ones, which relate to the same image side of the selected pixel. With reference to
In step 424, the system calculates an approximation of the level of the green color at the location of the selected pixel R(3,3). It is noted that the calculation can include a plurality of approximation procedures, such as calculating the weighted average level, depending on the location of pixel sensors G(5,1), G(3,2), G(3,5), G(5,4), and G(1,3), with respect to the location of the selected pixel sensor R(3,3). Similarly, blue color level at the location of the selected pixel sensor R(3,3), can be calculated using the information received from pixel sensors B(1,2), B(1,5), B(3,1), B(3,4) and B(5,3). Hence the disclosed technique provides a method for enhancing picture resolution by means of color information interpolation, using image processing.
It is noted that none of the lenticular elements is necessarily round shaped, but can be formed according to other optical structures which are based on various prism designs, and the like, which provide the directing of beams of light coming from different directions to different directions.
Reference is now made to
The sensor assembly 502 is generally similar to the sensor assembly 202, as described herein above in conjunction with
The stereoscopic display 514 includes two display units, a left display unit 516L (for placing in front of the left eye of the user), and a right display unit 516R (for placing in front of the right eye of the user). Hence, the stereoscopic display 514 is capable of displaying stereoscopic images continuously. A/D converter 528 converts analog information received from light sensor array 522 into digital format and provides the digital information to frame grabber 510.
The digital information is received by the frame grabber 510 and hence made available to the processor 508 via the bus. As the processor 508 processes the information, it uses the memory unit 504 as temporary storage. After processing the information, the processor 508 produces two matrices each being a reconstructed representation relating to one of the originally detected images. The processor 508 provides these matrices to the stereoscopic video generator 512, which in turn produces two respective video signals, one for the left view image and another for the right view image. The stereoscopic video generator 512 provides the video signals to the stereoscopic display 514, which in turn produces two images, one using right display unit 516R and another using left display unit 516L.
Reference is now made to
The super-pixel 550 includes four sensor sections, 554, 556, 558 and 560, arranged in a rectangular formation. The lenticular element 552 is shaped like a dome and is basically divided into four sections, each facing a different one of the sensor sections 554, 556, 558 and 560.
The super-pixel 550 and the lenticular element 552 form together, an optical detection unit, which is capable of detecting and distinguishing light which arrives from four different directions. The lenticular element 552 directs a portion of the upper-left side view of the detected object to sensor section 554 and directs a portion of the lower-left side view of the detected object to sensor section 556. In addition, the lenticular element 552 directs a portion of the upper-right side view of the detected object to sensor section 560 and a portion of the lower-right side view of the detected object to sensor section 558.
It is noted that according to a further aspect of the disclosed technique, the four-direction arrangement, which is described in
Reference is now made to
Reference is now made to
Reference is now made to
Detection apparatus 600 includes an optical assembly 602, a lenticular lens layer 604 and an array of sensors 608. The detection apparatus 600 detects images of an object 610, which includes a plurality of object sections 610A, 610B, 610C and 610D.
Sensor array 608 includes a plurality of super-pixels 608A, 608B, 608C and 608D. Each of these super-pixels is divided into a left-side section and a right-side section. For example, super-pixel 608A includes a left-side section, designated 608AL and a right-side section, designated 608AR.
The optical assembly 602 is divided into two optical sections 602L and 602R, each directed at transferring an image, which represents a different side view. Optical section 602R transfers an image, which is a view from the right side of object 610. Optical section 602L transfers an image, which is a view from the left side of object 610.
A plurality of light rays 612, 614, 616 and 618 are directed from all sections of the object 610 to the left side of optical assembly 602 (i.e., optical section 602L), and from there, are directed to the lenticular lens layer 604. Here, these rays are further directed to the left-side view associated sensor sections, which are sensor sections 608L (i.e., sensor sections 608AL, 608BL, 608CL and 608DL).
With reference to
Reference is now made to
In the present example, all of the light rays 646A, 646B, 646C and 646D arrive from object element 648A. Each of these rays is received at a different optical section. Ray 646A is received and directed by optical section 636, ray 646B is received and directed by optical section 638, ray 646C is received and directed by optical section 634 and ray 646D is received and directed by optical section 632. Each of the optical sections directs its respective ray to a specific lenticular element 642(1,1), at the right side of the lenticular lens layer 642. The location of lenticular element 642(1,1) is respective of the location of the object element 648A. The lenticular element 642(1,1) directs each of the rays to predetermined light sensors within its respective super-pixel 640(1,1).
In accordance with a further aspect of the disclosed technique, there is provided a reduced size color stereovision detection system, which uses time-multiplexed colored light projections, and respective time-multiplexed frame grabbing.
Reference is now made to
Device 650 includes a lenticular lens layer 652, a full spectrum sensor array 654, an optical assembly 660 and an illuminating unit 656, surrounding the optical assembly 660. Illuminating unit 656 includes a plurality of illuminating elements, generally referenced 658, each being of a specific predetermined color. Illuminating elements 658RED produce generally red light, illuminating elements 658GREEN produce generally green light and illuminating elements 658BLUE produce generally blue light. It is noted that each of the illuminating elements can be of a specific color (i.e., a specific wavelength), a range of colors (i.e., a range of wavelengths) or alternating colors, for example, a multi-color light emitting diode (LED).
Each group of illuminating elements, which are of the same color, is activated at a different point in time. For example, illuminating elements 658RED are activated and shut down first, illuminating elements 658GREEN are activated and shut down second and illuminating elements 658BLUE are activated and shut down last. Then the illuminating sequence is repeated.
With reference to
Hence, the first detected frame 664 in an illumination sequence is considered red, since the illuminating elements which were active at that time, were illuminating elements 658RED. Similarly, the second detected frame 666 in an illumination sequence is considered green, since the illuminating elements, which were active at that time, were illuminating elements 658GREEN. Finally, the last detected frame 668 in an illumination sequence is considered blue, since the illuminating elements, which were active at that time, were illuminating elements 658BLUE. It is noted that any other combination of colors is applicable for this and any other aspect of the disclosed technique, such as CYMG, and the like.
Reference is now made to
According to a further aspect of the disclosed technique, a remote multi-color light source 682 can be coupled with the narrow section 674 via additional light guiding members such as optic-fibers 684. Light, such as light ray 676, which is emitted from the light source 682, is directed via the light guiding members 684 to the narrow section 674. The light-guiding element 671 guides light ray 676, and projects it through the wide section 672. This arrangement is useful when using an external light source, which is to be placed outside the inspected area (for example, outside the body of the patient).
According to a further aspect of the disclosed technique, a full spectrum illumination unit, which produces white light, is combined with a device such as sensor assembly 202 (
Reference is now made to
Reference is now made to
In accordance with a further aspect of the disclosed technique there is provided a partial lenticular lens layer, which includes spaced apart lenticular elements. Reference is now made to
The partial lenticular lens layer 742 includes a plurality of lenticular elements designated 744(1), 744(2) and 744(3). Lenticular element 744(1) is located over the first two left columns of color sensors, generally referenced 746(1), of sensor array 740. Hence, the information received from these first two left columns of color sensors of sensor array 740 contains stereoscopic information. The third and fourth columns of color sensors, generally designated 746(2), of sensor array 740 do not have a lenticular element located thereon, and hence, cannot be used to provide stereoscopic information.
Similarly, lenticular element 744(2) and 744(3) are located over color sensor column pairs, 746(3) and 746(5), respectively, while color sensor column pairs, 746(4) and 746(6) are not covered with lenticular elements.
Reference is now made to
In accordance with a further aspect of the disclosed technique, there is provided a system, which produces a color stereoscopic image. The structure of the stereoscopic device defines at least two viewing angles, through which the detector can detect an image of an object. According to one aspect of the disclosed technique, the stereoscopic device includes an aperture for each viewing angle. Each of the apertures can be opened or shut. The stereoscopic device captures a stereoscopic image, by alternately detecting an image of an object, from each of the viewing angles, (e.g., by opening a different aperture at a time and shutting the rest) through a plurality of apertures, (at least two), each time from a different aperture. The final stereoscopic image can be reconstructed from the images captured with respect to the different viewing angles.
The detection of stereoscopic color image is provided by illuminating the object with a sequence of light beams, each at a different wavelength, and detecting a separate image for each wavelength and aperture combination.
Reference is now made to
System 800 includes a multiple aperture 804, a controller 834, an image detector 812, a storage unit 836, an image processor 838, a movement detector 814 and an illumination unit 830. The controller 834 is coupled with the multiple aperture 804, the image detector 812, the storage unit 836, movement detector 814 and to the illumination unit 830. The storage unit 836 is further coupled with the image processor 838. The multiple aperture 804 includes a plurality of apertures, generally referenced 802i, where each aperture can be activated to be open or closed. It is noted that when an aperture is open it is at least transparent to a predetermined degree to light, and when an aperture is closed, it substantially prevents the travel of light there through. Any type of controllable light valve can be used to construct each of the apertures. Movement detector 814 detects the movement of image detector 812. The detected movement can be a linear displacement, an angular displacement, and the derivatives thereof such as velocity, acceleration, and the like. The operation of system 800, according to data received from movement detector 814, is described herein below in connection with
Light valve elements are components, which have an ability to influence light in at least one way. Some of these ways are, for example: scattering, converging, diverging, absorbing, imposing a polarization pattern, influencing a polarization pattern which, for example, may be by rotation of a polarization plane. Other ways to influence light can be by influencing wave length, diverting the direction of a beam, for example by using digital micro-mirror display (also known as DMD) or by using field effect, influencing phase, interference techniques, which either block or transfer a portion of a beam of light, and the like. Activation of light valve elements, which are utilized by the disclosed technique, can be performed either electrically, magnetically or optically. Commonly used light valve elements are liquid crystal based elements, which either rotate or create and enforce a predetermined polarization axis.
In the present example, multiple aperture 804 includes two apertures 802R and 802L. The controller 834 further activates the multiple aperture 804, so as to alternately open apertures 802R and 802L. In
Light rays, which reflect from various sections of the object 810, pass through the currently open aperture (802R in
With reference to
The illumination unit 830 is a multi-color illumination unit, which can produce light at a plurality of wavelengths. The controller 834 provides a sequence of illumination commands to the illumination unit 830, so as to produce a beam at a different predetermined wavelength, at each given moment. In the present example, the illumination unit is a red-green-blue (RGB) unit, which can produce a red light beam, a green light beam and a blue light beam. It is noted that illumination unit 830 can be replaced with any other multi-color illumination unit, which can produce either visible light, non-visible light or both, at any desired wavelength combination (CYMG and the like).
Furthermore, illumination unit 830 can be a passive unit, where it receives external commands to move from one wavelength to another, or it can be an active unit, which changes wavelength independently and provides an indication of the currently active wavelength to an external controller. Illumination unit 830 of the present example is a passive unit, which enhances the versatility of the system 800, by providing any wavelength sequence on demand.
The image detector 812 includes a plurality of detection elements 808A, 808B, 808C and 808D. In accordance with one aspect of the disclosed technique, image detector 812 is a full range color detector, where each of the detection elements is operative to detect light in a plurality of wavelengths. In accordance with another aspect of the disclosed technique, the image detector 812 is a color segmented detector, where the detection elements are divided into groups, each operative to detect light in a different range of wavelengths. One conventional type of such detectors includes a full range detection array, which is covered by a color filter layer, where each detection element is covered by a different color filter. Accordingly, some of the detection elements are covered with red filters, others are covered with green filters and the rest are covered with blue filters.
The disclosed technique enhances the color resolution of systems, using such color detectors. It will be appreciated by those skilled in the art that a color segment detector of poor quality may exhibit a wavelength (color) overlap between the different detection elements. For example, when the filters are of poor quality, their filtering functions tend to overlap such as the red filter also passes a small amount of either green or blue light. Hence, the detection element behind the red filter, also detects that small amount of green or blue light, but provides an output measurement as a measurement of red light. Hence, the color detector produces an image, which includes incorrect measurements of red light (e.g. more than the actual red light, which arrived at the detector) as result of that overlap. Accordingly, received information of the inspected object is not valid.
In the disclosed technique, the illumination unit 830 produces a sequence of non-overlapping illumination beams at predetermined wavelengths (i.e., red, blue and green). As explained above, the color detector detects an image, which includes incorrect measurements, as a result of the wavelength (color) filtering overlap. Since the illumination unit 830 and the image acquisition process are synchronized, the imaging system can process each of the acquired images, according to the actual light beam color, which was produced therewith. For example, the illumination unit 830 produces blue light illumination beam. At the same time the image detector 812 detects an image, which also includes actual light measurements in detection elements, which are covered with green and red filters, due to the wavelength overlap. The imaging system can discard light measurements, which are received from detection elements, covered with color filters, which are not blue (e.g., red and green).
Such sequenced color illumination of the object, provides enhanced color resolution, for color image detectors of poor quality, and obtains the valid color images of the inspected object. System 800 can further include a stereoscopic display unit (not shown), coupled with controller 834 for displaying a stereoscopic image of object 810.
Reference is further made to
Timing sequence 841 rises every time any of the rises of sequences 844, 846 and 848 intersect with a rise of either sequence 842 or sequence 840. For example, rise 841A indicates a frame download of a blue light—right aperture combination, rise 841B indicates a frame download of a green light—right aperture combination, and rise 841C indicates a frame download of a red light—right aperture combination. Similarly, rise 841D indicates a frame download of a blue light—left aperture combination, rise 841E indicates a frame download of a green light—left aperture combination and rise 841F indicates a frame download of a red light—left aperture combination.
It is noted that for some light sources, the produced light beams do not cover the full range of visible light. For such light sources, the missing color components can be reconstructed (interpolated) taking into consideration the physiological assumption, that color reflection response as a function of reflected angle, does not change much with angle.
Reference is further made to
In step 872 right and left apertures are alternated. Light rays, which reflect from various sections of the object 810, pass through the currently open aperture (802R in
In step 874, a plurality of frames, each for a selected aperture and wavelength combination is detected. Controller 834 operates the image detector 812 so as to detect a plurality of frames, each respective of a selected aperture and wavelength combination.
Light rays 822 and 824 (
Light rays 827 and 825 (
With reference to
In step 876, movement between the detector and the inspected organ, at selected frequencies is detected. This movement can be detected from movement of the endoscope, by means of a movement detector, or by analyzing the detected images, where different color images exhibit different lines, with dramatic color shade changes. This information is utilized in the following step, for spatially correlating between images of different colors.
In step 878 a stereoscopic color image from the plurality of frames, according to their aperture origin is produced. With reference to
Reference is further made to
Timing sequence 841′ rises every time any of the rises of sequence 844′, 846′ and 848′ intersects with a rise of either sequence 842′ or sequence 840′. For example, rise 841′A indicates a frame download of a blue light—right aperture combination, rise 841′B indicates a frame download of a blue light—left aperture combination and rise 841′C indicates a frame download of a green light—right aperture combination. Similarly, rise 841′D indicates a frame download of a green light—left aperture combination, rise 841′E indicates a frame download of a red light—right aperture combination and rise 841′F indicates a frame download of a blue light—left aperture combination.
Reference is further made to
Reference is further made to
It is noted that a mechanical multi-wavelength illumination unit such as described in the prior art, can be used for implementing the disclosed technique. However, such a system significantly reduces the capability of the user to control illumination duration, wavelength ratio and detection timing, such as described herein above.
The disclosed technique incorporates even more advanced aspects, which provide automatic image translation correction, based on correlation between the two detected images. When the endoscope is handheld, it is subjected to the vibration of the human hand, which is in the order of 10 Hz, at an angular amplitude of 1 degree. This phenomenon causes a blur of areas, where different colors intersect, and is also known as the “between color field blur” effect. It is noted that any movement between the image detector and the inspected organ can cause this phenomenon, provided it occurs at particular frequencies, defined by the structure and the manner of operation of the system.
With reference to
Reference is now made to
The foregoing description relates to one aspect of the disclosed technique, in which a stereoscopic image of an object is captured by a sensor array through a lenticular lens layer (i.e., each captured image includes all the primary colors of the color palette, such as RGB, CYMG, and the like). It is noted that the movement is determined such that it has a constant average (e.g., vibrating about a certain point).
With reference to
With reference to X1 to the right of point O1 and a distance
Y1 below point O1. In this case the length of
X1 is equal to the horizontal width of two pixels of detected image 780, and the length
Y1 is equal to the vertical height of minus two pixels of detected image 780. Movement detector 230 detects the movement of sensor assembly 768 from point O1 to point O2, and sends a signal respective of this movement, to processor 208.
With reference to
With reference to X2 to the left of point O1 and a distance
Y2 above point O1. In this case the length of
X2 is equal to the horizontal of minus two pixels of detected image 782, and the length
Y2 is equal to the vertical height of one pixel of detected image 782. Movement detector 230 detects the movement of sensor assembly 768 from point O1 to point O3, and sends a signal respective of this movement, to processor 208.
With reference to
It is noted that processor 208 processes the detected images 780 and 782, if the dimensions X1,
X2,
Y1 and
Y2 are of the order of A, the amplitude of vibrations of the human hand and in the appropriate frequency. In general, processor 208 performs the compensation process, between a plurality of captured images, as long as the detected movement, is maintained about a certain average point (XAVERAGE,YAVERAGE). When one of the average values XAVERAGE and YAVERAGE changes, then processor 208 initiates a new compensation process around the updated average point, accordingly.
Reference is now made to
Image processor 838 (
In the following description, object 810 (
Furthermore, the color of detected images 1052, 1054, 1056, 1058, 1060, and 1062 changes as described herein above for example in connection with
While multiple aperture 804 switches to aperture 802L, the center of sensor assembly 768 (
According to
For example, image processor 838 selects sub-matrices 1064, 1068 and 1072 (
It is noted that an RGB timing scheme can be employed. In this case, the stereoscopic display displays the sub-matrices in a sequence of right-red, left-green, right-blue, left-red, right-green and left-blue.
It is noted that the sequence of
According to another aspect of the disclosed technique, the locations from which the three-dimensional object is viewed from the right side and from the left side thereof, are further separated. Thus, the difference between the right side view image and the left side view image is substantially increased and the stereoscopic notion produced by the two images is substantially enhanced.
Reference is now made to
Apparatus 1100 includes a periscopic assembly 1102, an optical assembly 1104, a lenticular lens layer 1106 and a light sensor array 1108. Periscopic assembly 1102 includes a right mirror 1110, a left mirror 1112, a right center mirror 1114 and a left center mirror 1116. Lenticular lens layer 1106 and light sensor array 1108 are similar to lenticular lens layer 104 and light sensor array 102, respectively, as described herein above in connection with
With reference to
Left mirror 1112 receives a light beam 1122A, which is a left side view of the right side of three-dimensional object 1118. Left mirror 1112 reflects light beam 1122A, as a light beam 1122B. Left center mirror 1116 reflects light beam 1122B toward optical assembly 1104, as a light beam 1122C. Optical assembly 1104 directs a light beam 1122D to lenticular element 1128 of lenticular lens layer 1106. Lenticular element 1128 focuses light beam 1122D on a sensor 1132 of light sensor array 1108.
With reference to
Right mirror 1110 receives a light beam 1126A, which is a right side view of the left side of three-dimensional object 1118. Right mirror 1110 reflects light beam 1126A, as a light beam 1126B. Right center mirror 1114 reflects light beam 1126B toward optical assembly 1104, as a light beam 1126C. Optical assembly 1104 directs a light beam 1126D to lenticular element 1134 of lenticular lens layer 1106. Lenticular element 1134 focuses light beam 1126D on a sensor 1138 of light sensor array 1108.
It is noted that right mirror 1110 and right center mirror 1114 together operate similar to a periscope. Likewise, left mirror 1112 and left center mirror 1116 together operate similar to a periscope. Right mirror 1110 and left mirror 1112 are located substantially apart relative to an axis which is perpendicular to lenticular lens layer 1106 and which passes through the junction of right center mirror 1114 and left center mirror 1116. Hence, right mirror 1110 detects a right side view of three-dimensional object 1118, which is substantially different than the left side view thereof, detected by left mirror 1112. Thus, the respective light detecting elements of light sensor array 1108 receive light beams respective of the right side view and the left side view of three-dimensional object 1118, which are more distinct than in the case of
According to another aspect of the disclosed technique, a light valve alternately differentiates between images of a three-dimensional object received from different directions, and alternately provides these images to an image detector. Thus, the image detector alternately detects images of the three-dimensional object, from different sides thereof.
Reference is now made to
Apparatus 1150 includes a periscope assembly 1152, a multiple aperture 1154, an optical assembly 1156, a light sensor array 1158, a controller 1160, a storage unit 1162 and an image processor 1164. Periscope assembly 1152 includes a right mirror 1166, a left mirror 1168, a right center mirror 1170 and a left center mirror 1172. Multiple aperture 1154 includes a right aperture 1174R and a left aperture 1174L. Multiple aperture 1154 is similar to multiple aperture 804, as described herein above in connection with
Periscope assembly 1152 is located between a three-dimensional object 1176 and multiple aperture 1154. Multiple aperture 1154 is located between periscope assembly 1152 and optical assembly 1156. Multiple aperture 1154 is located substantially close to periscope assembly 1152. Optical assembly 1156 is located between multiple aperture 1154 and light sensor array 1158. Multiple aperture 1154, light sensor array 1158, controller 1160, storage unit 1162 and image processor 1164, are interconnected via a bus 1186. Controller 1160 controls multiple aperture 1154, such that right aperture 1174R and left aperture 1174L alternately open and close.
With reference to
With reference to
Alternatively, multiple aperture 1154 is located between three-dimensional object 1176 and periscope assembly 1152. In this case, right mirror 1166 receives a right side view image of three-dimensional object 1176 only when right aperture 1174R is open. Similarly, left mirror 1168 receives the left side view image of three-dimensional object 1176, only when left aperture 1174L is open. Multiple aperture 1154 is located substantially close to periscope assembly 1152.
Alternatively, an illuminator similar to illuminator 830 (
It is noted that the illuminator can emit light in the visible range of wavelengths, as well as in the invisible range of wavelengths. In addition, the wavelength of light emitted by the illuminator can be generally discrete (e.g., green light is emitted either at 500 nm, 525 nm, 542 nm, and so on).
According to another aspect of the disclosed technique, image differentiation is performed sequentially by filtering light at different sets of wavelengths for each of the right side image and the left side image. According to one embodiment two different light filters, a right side filter and a left side filter, are placed between a three-dimensional object and an image detector. The right side filter admits light at one set of ranges of wavelengths and the left side filter admits light at another set of ranges of wavelengths. The two sets of ranges of wavelengths are mutually exclusive. The right side filter receives a right side view image of the three-dimensional object and the left side filter receives a left side view image of the three-dimensional object.
The three-dimensional object is sequentially illuminated with two groups of wavelengths. The first group of wavelengths is included only in the set of ranges of wavelengths of right side filter. The second group of wavelengths is included only in the set of ranges of wavelengths of the left side filter.
When the object is illuminated with first group of wavelengths, the right side filter passes a right side image to the image detector, while the left side filter blocks these wavelengths. Similarly, when the object is illuminated with second group of wavelengths, the left side filter passes a left side image to the image detector, while the right side filter blocks these wavelengths.
Reference is now made to
Apparatus 1200 includes a right side filter 1202, a left side filter 1204, a periscope assembly 1206, an optical assembly 1208, a light sensor array 1210, an illuminating unit 1240, a controller 1216, a storage unit 1218 and an image processor 1220. Periscope assembly 1206 includes a right mirror 1222, a left mirror 1224, a right center mirror 1226 and a left center mirror 1228. Illuminating unit 1240 includes illuminators 1212 and 1214. Right side filter 1202 is a light filter, which admits light only in red, green and blue ranges of wavelengths ΔR1, ΔG1 and ΔB1, respectively. Left side filter 1204 is a light filter which admits light only in red, green and blue ranges of wavelengths ΔR2, ΔG2 and ΔB2, respectively, where the ranges of wavelengths ΔR1, ΔG1 and ΔB1 and the ranges of wavelengths ΔR2, ΔG2 and ΔB2 do not overlap. Illuminator 1212 emits light at the group of wavelengths R1, G1 and B1 (i.e., RGB1), which are is included in the ranges of wavelengths ΔR1, ΔG1 and ΔB1 and excluded from the ranges of wavelengths ΔR2, ΔG2 and ΔB2. Illuminator 1214 emits light at the group of wavelengths R2, G2 and B2 (i.e., RGB2), which is included in the ranges of wavelengths ΔR2, ΔG2 and ΔB2 and excluded from the ranges of wavelengths ΔR1, ΔG1 and ΔB1. Thus, illuminating unit 1240 sequentially emits light at the group of wavelengths RGB1 and RGB2. It is noted that R1 refers to one wavelength or more, which are included in the red wavelength range R, arranged continuously, discretely or in a mixed fashion. The same applies to R2 with respect to R, G1 and G2 with respect to the green wavelength range G and B1 and B2 with respect to the blue wavelength range B. This applies to all types of wavelength differentiators which shall be disclosed further below.
In the example set forth in
Right side filter 1202 and left side filter 1204 are located between a three-dimensional object 1230 and periscope assembly 1206. Optical assembly 1208 is located between periscope assembly 1206 and light sensor array 1210. Light sensor array 1210, controller 1216, storage unit 1218 and image processor 1220 are interconnected via a bus 1268. Illuminating unit 1240 is coupled with controller 1216.
With reference to
Right center mirror 1226 reflects the reflection of light beams 1232 and 1234 from right mirror 1222, to optical assembly 1208. Optical assembly 1208 focuses light beams 1232 and 1234 on light sensor array 1210. Thus, when illuminating unit 1240 emits light at the group of wavelengths RGB1 the right side view image of three-dimensional object 1230 at the group of wavelengths RGB, reaches light sensor array 1210. It is noted that since the group of wavelengths RGB, is not included in any of the ranges of wavelengths at which left side filter 1204 admits light, left side filter 1204 blocks light beams 1236 and 1238, and that the left side view image of three-dimensional object 1230 does not reach light sensor array 1210 at this stage. Controller 1216 stores this right side view image of three-dimensional object 1230, in storage unit 1218.
With reference to
Left center mirror 1228 reflects the reflection of light beams 1264 and 1266 from left mirror 1224, to optical assembly 1208. Optical assembly 1208 focuses light beams 1264 and 1266 on light sensor array 1210. Thus, when illuminating unit 1240 emits light at the group of wavelengths RGB2 the left side view image of three-dimensional object 1230 at the group of wavelengths RGB2 reaches light sensor array 1210. Since the group of wavelengths RGB2 is not included in any of the ranges of wavelengths at which right side filter 1202 admits light, right side filter 1202 blocks light beams 1260 and 1262, and the right side view image of three-dimensional object 1230 does not reach light sensor array 1210 at this stage. Controller 1216 stores this left side view image of three-dimensional object 1230, in storage unit 1218.
Image processor 1220 retrieves the right side and the left side view images of three-dimensional object 1230, from storage unit 1218 and produces stereoscopic images of three-dimensional object 1230, by processing the right side and the left side view images. It is noted that in the example set forth in
Alternatively, in a system which includes a full range light sensor array, the controller controls the operation of the illuminating unit, to sequentially emit light at individual groups of wavelengths R1, R2, G1, G2, B1 and B2. In this case, the right side filter admits a sequence of right side view images of the three-dimensional object, in each of the ranges of wavelengths R1, G1 and B1, and then the left side filter admits a sequence of left side view images of the three-dimensional object, in each of the wavelengths R2, G2 and B2. For each cycle in the illumination sequence, the controller enables the light sensor array to detect six images of the three-dimensional object. Three of these images are right side view images, each at a different one of the groups of wavelengths R1, G1 and B1. The other three images are left side view images, each at a different one of the groups of wavelengths R2, G2 and B2. It is noted that other sequences of R1, R2, G1, G2, B1 and B2, as well as other divisions of light (e.g., CYMG, and CYMG2) are applicable.
In the example set forth in
Alternatively, the right side filter and the left side filter are located between the periscope assembly and the optical assembly. In this case, the right side filter receives a right side view image of the three-dimensional object from the right center mirror, and the left side filter receives a left side view image of the three-dimensional object from the left center mirror.
Alternatively, a rotating disk is placed in front of the periscope assembly and an illuminator constantly emits light. Half of the rotating disk is transparent and the other half is opaque. Thus, as the rotating disk rotates, the periscope assembly alternately receives the right side and the left side view images of the three-dimensional object and directs these images to the light sensor array.
With reference to
When the transparent portion of the partially-transparent rotating disk is located above the right mirror, the right mirror receives a right side view image of the three-dimensional object and the opaque portion of the partially-transparent rotating disk blocks the light to the left mirror. When the transparent portion of the partially-transparent rotating disk is located above the left mirror, the left mirror receives a left side view image of the three-dimensional object and the opaque portion of the partially-transparent rotating disk blocks the light to the right mirror. The controller enables the light sensor array to alternately detect a right side view image and a left side view image of the three-dimensional object, according to the position of the transparent portion relative to the right mirror and the left mirror. The controller alternately stores the right side view images and the left side view images in the storage unit. The image processor concurrently retrieves the right side view images and left side view images of the three-dimensional object, processes these images and provides a respective video signal to a stereoscopic display, such as stereoscopic display 214 (
Alternatively, a rotating disk is placed in front of the periscope assembly and a multi-wavelength illuminator sequentially emits light in different ranges of wavelengths. Half of the rotating disk is transparent and the other half is opaque. As the rotating disk rotates, the periscope assembly receives a sequence of right side and left side view images of the three-dimensional object, in different ranges of wavelengths and directs these images to the light sensor array. This embodiment is similar to the embodiments described herein above in connection with
With reference to
For example, when the multi-wavelength illuminator sequentially illuminates the three-dimensional object in red, green and blue (i.e., RGB), and the transparent portion is located above the right mirror, the light sensor array detects a sequence of images in red, green and blue. According to the position of the partially-transparent rotating disk and the state of the multi-wavelength illuminator, the controller determines that these images are right side view images of the three-dimensional object, in red, green and blue, respectively. The controller stores these images in the storage unit.
The light sensor array detects right side view images when the transparent portion is located above the right mirror. The light sensor array detects left side view images when the transparent portion is located above the left mirror. The controller tags each of these images according to the state of multi-wavelength illuminator (e.g., red, green and blue) at the time when each of these images was captured. According to a simple setting, at a given time period, the stereoscopic imaging apparatus produces six images, three for each side, two for each color (e.g., a left side blue image, a left side green image, a left side red image, a right side blue image, a right side green image and a right side red image).
Alternatively, a rotating disk having an opaque portion and a multi-wavelength transparent portion, is placed in front of the periscope assembly and an illuminator illuminates the three-dimensional object. As the rotating disk rotates, the periscope assembly receives a sequence of right side and left side view images of the three-dimensional object, in different ranges of wavelengths and directs these images to the light sensor array. This embodiment is similar to the embodiments described herein above in connection with
With reference to
The illuminator provides light at least in the predetermined ranges of wavelengths as defined by the filtering sectors. As the multi-wavelength rotating disk rotates, the light sensor array detects a sequence of images. The controller determines the type of each of these images (i.e., either right side view image or left side view image) and the range of wavelengths of each of these images, according to the position of the multi-wavelength rotating disk.
According to another aspect of the disclosed technique, a pair of polarizers direct an image from one side of a three-dimensional object to an image detector, when both polarizers are oriented at the same angle, while another pair of polarizers block an image from another side of the object, when the polarizers are oriented 90 degrees apart. The relative polarization angles between the two polarizers in each pair is alternately changed to be either zero or 90 degrees. Thus, the image detector alternately receives images from different sides of the three-dimensional object.
Reference is now made to
Apparatus 1300 includes a periscope assembly 1302, a right polarizer 1304, a left polarizer 1306, a main polarizer 1308, an optical assembly 1310, a light sensor array 1312, a controller 1314, a storage unit 1316 and an image processor 1318. Periscope assembly 1302 includes a right mirror 1320, a left mirror 1322, a right center mirror 1324 and a left center mirror 1326.
Each of right polarizer 1304, left polarizer 1306 and main polarizer 1308 is an optical element which admits light only at a predetermined direction of polarization. In the following example, the polarization angle of the incident light beam is zero degrees, and the polarizer is rotated by 45 degrees relative to this polarization angle. The light vector, having a length of L and being set at zero angle, can be described as a vectorial combination of two vectors, each at a length √{square root over (2)} L, one directed at 45 degrees and the other directed at −45 degrees. The polarizer admits the vector which is directed at 45 degrees and blocks the vector which is directed at −45 degrees. The polarization angle of a polarizer can be changed electronically. The polarization angle of right polarizer 1304 and left polarizer 1306 is fixed, whereas the polarization angle of main polarizer 1308 can be changed. In the example set forth in
With reference to
In the example set forth in
Optical assembly 1310 focuses light beams 1330, 1332, 1334 and 1336 on light sensor array 1312. Since the polarization angles of right polarizer 1304 and left polarizer 1306 are approximately 90 degrees apart, main polarizer 1308 blocks light beams 1334 and 1336. Since the polarization angle of main polarizer 1308 is approximately the same as that of right polarizer 1304, main polarizer 1308 passes light beams 1330 and 1332 toward light sensor array 1312. Controller 1314 enables light sensor array 1312 to detect a right side view image of three-dimensional object 1328, according to the polarization angle main polarizer 1308. Controller 1314 stores this right side view image in storage unit 1316.
With reference to
Alternatively, a rotating polarizing disk replaces right polarizer 1304 and left polarizer 1306. The rotating polarizing disk is divided to two polarizing sections. The polarization angle of the first section is substantially equal to the polarization angle of the main polarizer and the polarization angle of the second section is away from the polarization angle of the main polarizer, by substantially 90 degrees. It is noted that certain limitations may apply to such a rotating polarizing disk, whereas the polarizers on the disk physically rotate. Accordingly, the rotating polarizing disk may include dynamic polarizers, which change according to the angular position of the rotating polarizing disk. Alternatively, the rotating polarizing disk is stopped or slowed down at predetermined angular positions, when an image is acquired.
It is noted that different structures of polarizers can be used for separating the images. Such structures include active and passive polarizers, located at various positions such as between the object and the periscope assembly, between the periscope assembly and the optical assembly and between the optical assembly and the light sensor array. The following are mere examples for such structures of polarizes.
Alternatively, main polarizer 1308 is located between three-dimensional object 1328 and periscope assembly 1302, while right polarizer 1304 and left polarizer 1306 are located between periscope assembly 1302 and optical assembly 1310. Further alternatively, right polarizer 1304 and left polarizer 1306 are located between three-dimensional object 1328 and periscope assembly 1302, while main polarizer 1308 is located between periscope assembly 1302 and optical assembly 1310.
Yet further alternatively, right polarizer 1304 and left polarizer 1306 are located between three-dimensional object 1328 and periscope assembly 1302, while main polarizer 1308 is located between optical assembly 1310 and light sensor array 1312. Still further alternatively, main polarizer 1308 is located between periscope assembly 1302 and optical assembly 1310, while right polarizer 1304 and left polarizer 1306 are located between optical assembly 1310 and light sensor array 1312. Yet further alternatively, main polarizer 1308 is located between three-dimensional object 1328 and periscope assembly 1302, while right polarizer 1304 and left polarizer 1306 are located between optical assembly 1310 and light sensor array 1312.
Still further alternatively, right polarizer 1304 and left polarizer 1306 are located between three-dimensional object 1328 and main polarizer 1308. Main polarizer 1308 is located between right polarizer 1304 and left polarizer 1306 on one side and periscope assembly 1302 on the other side.
Yet further alternatively, main polarizer 1308 is located between three-dimensional object 1328 on one side and right polarizer 1304 and left polarizer 1306 on the other side. Right polarizer 1304 and left polarizer 1306 are located between main polarizer 1308 and periscope assembly 1302.
Still further alternatively, right polarizer 1304 and left polarizer 1306 are located between periscope assembly 1302 and main polarizer 1308. Main polarizer 1308 is located between right polarizer 1304 and left polarizer 1306 on one side and optical assembly 1310 on the other side.
Yet further alternatively, main polarizer 1308 is located between periscope assembly 1302 on one side and right polarizer 1304 and left polarizer 1306 on the other side. Right polarizer 1304 and left polarizer 1306 are located between main polarizer 1308 and optical assembly 1310.
Still further alternatively, right polarizer 1304 and left polarizer 1306 are located between optical assembly 1310 and main polarizer 1308. Main polarizer 1308 is located between right polarizer 1304 and left polarizer 1306 on one side and light sensor array 1312 on the other side.
Yet further alternatively, main polarizer 1308 is located between optical assembly 1310 on one side and right polarizer 1304 and left polarizer 1306 on the other side. Right polarizer 1304 and left polarizer 1306 are located between main polarizer 1308 and light sensor array 1312.
Further alternatively, the polarization angle of main polarizer 1308 is fixed and the polarization angle of right polarizer 1304 and left polarizer 1306 can be changed. In this case, controller 1314 is coupled with right polarizer 1304 and left polarizer 1306 instead of main polarizer 1308 and hence, controller 1314 controls the angle of both right polarizer 1304 and left polarizer 1306. The polarization angles of right polarizer 1304 and left polarizer 1306 are changed substantially simultaneously and alternately by substantially 90 degrees each time, while the angle there between is substantially 90 degrees at all times.
According to another aspect of the disclosed technique, the image differentiator includes a combination of polarizers and polarization rotating cells. Each polarization rotating cell sequentially changes the polarization angle of light which exits each of two polarizers.
According to one embodiment, the image differentiator includes a front right polarizer, a front left polarizer, a polarization rotating cell and a main polarizer. The front right polarizer and the front left polarizer are located in the right channel and the left channel, respectively. The polarization rotating cell is located in the common path. The main polarizer is located in the common path between the polarization rotating cell and the light sensor array. The polarization angle of the front right polarizer is substantially equal to the polarization angle of the main polarizer, while the polarization angle of the front left polarizer is approximately 90 degrees away from that of the main polarizer. The polarization rotating cell receives light from both the front right polarizer and the front left polarizer. The polarization rotating cell is coupled with the controller.
A polarization rotating cell is generally in form of a crystal which changes the polarization angle of the incoming light by a selected value. In the present example, the polarization rotating cell alternates between two states. At the first state, the polarization rotating cell rotates any light incident thereon by a zero angle, thereby leaving the polarization angle of that incident light, unchanged. At the second state, the polarization rotating cell rotates any light incident thereon, by a substantially right angle (i.e., 90 degrees).
When the polarization rotating cell is in the first state, the polarization rotating cell leaves the polarization of the light exiting the front right polarizer and the front left polarizer unchanged. Since the polarization of the front right polarizer is substantially equal to the polarization of the main polarizer, the main polarizer admits the light which previously exited the front right polarizer. Since the polarization of the front left polarizer is substantially rotated at 90 degrees away from the polarization of the main polarizer, the main polarizer blocks the light which previously exited the front left polarizer. Thus, the main polarizer admits the right side view image of the three-dimensional object to the light sensor array, while the main polarizer blocks the left side view image of the three-dimensional object.
When the polarization rotating cell is in the second state, the polarization rotating cell rotates the polarization of the light received from the front right polarizer and from the front left polarizer, by substantially 90 degrees. In this case, the polarization of the light which previously exited the front left polarizer, is rotated to be substantially equal to the polarization angle of the main polarizer. Furthermore, the polarization of the light which exited the front right polarizer is rotated to be at substantially 90 degrees away from the polarization of the main polarizer. The main polarizer admits the light which previously exited the front left polarizer, while the main polarizer blocks the light which previously exited the front right polarizer. Thus, the main polarizer admits the left side view image of the three-dimensional object to the light sensor array, while the main polarizer blocks the right side view image of the three-dimensional object. The controller enables the light sensor array to detect the right side view image and the left side view image of the three-dimensional object, according to the rotating state of the polarization rotating cell.
According to another embodiment, a right polarization rotating cell is located between the front right polarizer and the main polarizer, in the right channel and a left polarization rotating cell is located between the front left polarizer and the main polarizer, in the left channel. The main polarizer is located in the common path, between the right polarization rotating cell and the left polarization rotating cell on one side and the light sensor array on the other. The front right polarizer, the front left polarizer and the main polarizer are static polarizers. The polarization angles of the front right polarizer, the front left polarizer and the main polarizer are substantially equal. The right polarization rotating cell and the left polarization rotating cell are coupled with the controller, which alternately provides two states of operation.
In the first state of operation, the controller sets the rotation angle of the right polarization rotating cell to zero degrees and the rotation angle of the left polarization rotating cell to 90 degrees. Accordingly, the polarization of the light which previously exited the front right polarizer remains substantially unchanged, while the polarization of the light which previously exited the front left polarizer is changed by a substantially right angle. The main polarizer admits the light which previously exited the front right polarizer, while the main polarizer blocks the light previously exited the front left polarizer. Thus, the main polarizer admits the right side view image of the three-dimensional object to the light sensor array, while blocking the left side view image of the three-dimensional object.
In the second state of operation, the controller sets the rotation angle of the left polarization rotating cell to zero degrees and the rotation angle of the right polarization rotating cell to 90 degrees. Accordingly, the polarization of the light which previously exited the front left polarizer remains substantially unchanged, while the polarization of the light which previously exited the front right polarizer is changed by substantially 90 degrees. The main polarizer admits the light which previously exited the front left polarizer, while the main polarizer blocks the light previously exited the front right polarizer. Thus, the main polarizer admits the left side view image of the three-dimensional object to the light sensor array, while the main polarizer blocks the right side view image of the three-dimensional object. The controller enables the light sensor array to detect the right side view image and the left side view image of the three-dimensional object, according to the rotating states of the right polarization rotating cell and the left polarization rotating cell.
According to another embodiment, the main polarizer is eliminated, the front right polarizer and the front left polarizer are static polarizers and the polarization angle of the front right polarizer is substantially 90 degrees away from the polarization angle of the front left polarizer. In addition a polarized light source is employed, which is coupled with the controller. The polarized light source alternately illuminates the three-dimensional object with light at a first polarization angle and at a second polarization angle. The first polarization angle of the illuminating light is substantially equal to the polarization angle of the front right polarizer and the second polarization angle of the illuminating light is substantially equal to the polarization angle of the front left polarizer.
When the polarized light source illuminates the three-dimensional object at the polarization angle of the front right polarizer, the periscope assembly directs the right side view image of the three-dimensional object to the front right polarizer, substantially at the polarization angle of the front right polarizer. Simultaneously, the periscope assembly directs the left side view image of the three-dimensional object to the front left polarizer, substantially at the polarization angle of the front right polarizer. Since the polarization angle of the right side view image is substantially equal to the polarization angle of the front right polarizer, the front right polarizer admits the right side view image of the three-dimensional object to the light sensor array, through the optical assembly. Since the polarization angle of the left side view image is substantially 90 degrees away from the polarization angle of the front left polarizer, the front left polarizer blocks the left side view image of the three-dimensional object.
When the polarized light source illuminates the three-dimensional object at the polarization angle of the front left polarizer, the periscope assembly directs the left side view image of the three-dimensional object to the front left polarizer, substantially at the polarization angle of the front left polarizer. Simultaneously, the periscope assembly directs the right side view image of the three-dimensional object to the front right polarizer, substantially at the polarization angle of the front left polarizer. Since the polarization angle of the left side view image is substantially equal to the polarization angle of the front left polarizer, the front left polarizer admits the left side view image of the three-dimensional object to the light sensor array, through the optical assembly. Since the polarization angle of the right side view image is substantially 90 degrees away from the polarization angle of the front right polarizer, the front right polarizer blocks the left side view image of the three-dimensional object. The controller enables the light sensor array to detect the right side view image and the left side view image of the three-dimensional object, according to the illuminating state of the polarized light source.
It is noted that in this case, the three-dimensional object is illuminated only with light at a selected polarization angle at each state of the polarized light source. Thus, the three-dimensional object is heated substantially less and the physical properties thereof remain substantially stable.
Reference is now made to
In step 1402, the two received images are directed to a common path. With reference to
In step 1404, the two received images are differentiated. With reference to
According to another embodiment of the disclosed technique, the periscope assembly moves between a retracted position and an extended position. Thus, the endoscope is entered into the body of the patient while the periscope assembly is retracted, thereby assuming a narrow shape, capable of entering through narrow passages. When the endoscope is located in a selected region within the body of the patient, the periscope assembly moves to an extended position, thereby separating apart the apertures which receive a right side view and a left side view of the selected region. The periscope, then transfers substantially distinct right side view and left side view images of the selected region, to an image detector via an optical assembly.
Reference is now made to
Endoscope 1450 includes a periscope assembly 1452, an optical assembly 1454, a lenticular lens layer 1456, and a light sensor array 1458. Periscope assembly 1452 includes a right mirror 1460, a left mirror 1462, a right center mirror 1464, a left center mirror 1466, and hinges 1468, 1470 and 1472. Optical assembly 1454 includes a plurality of lenses 1474 and 1476.
Periscope assembly 1452 is located at a distal end 1478 of endoscope 1450. Optical assembly 1454 is located between periscope assembly 1452 and lenticular lens array 1456. Lenticular lens array 1456 is located between optical assembly 1454 and light sensor array 1458. Right mirror 1460 and left mirror 1462 can rotate about hinges 1468 and 1470, respectively. Right center mirror 1464 and left center mirror 1466 can rotate about hinge 1472.
With reference to
Reference is now made to
Endoscope 1500 includes a periscope assembly 1502, an optical assembly 1504, a lenticular lens layer 1506, and a light sensor array 1508. Periscope assembly 1502 includes a right prism 1510, a left prism 1512, a hinge 1514, a rail 1516 and a stop 1518. Optical assembly 1504 includes a plurality of lenses 1520 and 1522. Each of right prism 1510 and left prism 1512 is a prism whose longitudinal cross section is a parallelogram. Right prism 1510 and left prism 1512 can rotate about hinge 1514. Hinge 1514 can slide within rail 1516 in directions designated by arrows 1524 and 1526. Stop 1518 is coupled with rail 1516. Periscope assembly 1502 is located at a distal end 1528 of endoscope 1500. Optical assembly 1504 is located between periscope assembly 1502 and lenticular lens layer 1506. Lenticular lens layer 1506 is located between optical assembly 1504 and light sensor array 1508.
With reference to
Optical assembly 1504 focuses the right side view image and the left side view image of three-dimensional object 1538, on lenticular lens layer 1506. Lenticular lens layer 1506 differentiates between the right side view image and the left side view image, and the respective detection elements of light sensor array 1508 detect the right side view image and the left side view image of three-dimensional object 1538. When hinge 1514 moves in direction 1526, surfaces 1530 and 1532 make contact with stop 1518 and right prism 1510 and left prism 1512 move back to the retracted position of
Reference is now made to
Apparatus 1560 includes a periscope assembly 1562, an optical assembly 1564, a light sensor array 1566, a controller 1568, a storage unit 1570 and an image processor 1572. Periscope assembly 1562 includes a right mirror 1574, a left mirror 1576, a rotating mirror 1578 and a hinge 1580. One side of rotating mirror 1578 is reflective and the other side thereof is non-reflective. Periscope assembly 1562 is located between a three-dimensional object 1582 and optical assembly 1564. Optical assembly 1564 is located between periscope assembly 1562 and light sensor array 1566. Hinge 1580 is coupled with a moving element (not shown), such as a piezoelectric element, a pulling force of a cable against a spring, and the like. The moving element, light sensor array 1566, storage unit 1570 and image processor 1572 are interconnected via a bus 1592.
Right mirror 1574 is oriented at a slanted angle with respect to three-dimensional object 1582 and at the right side of three-dimensional object 1582, such that right mirror 1574 receives a right side view image of three-dimensional object 1582. This slanted angle is preferably close to 45 degrees. Left mirror 1576 is oriented at another slanted angle, opposite to the slanted angle of right mirror 1574 and at the left side of three-dimensional object 1582. Left mirror 1576 receives a left side view image of three-dimensional object 1582.
The moving element alternately rotates rotating mirror 1578 about hinge 1580, between two positions. At one position, rotating mirror 1578 is oriented at an angle substantially parallel to the slanted angle of right mirror 1574. In this position, the reflective side of rotating mirror 1578 faces right mirror 1574 while the non-reflective side of rotating mirror 1578 faces left mirror 1576. At another position, rotating mirror 1578 is oriented at an angle substantially parallel to the slanted angle of left mirror 1576. In this position, the reflective side of rotating mirror 1578 faces left mirror 1576 while the non-reflective side of rotating mirror 1578 faces right mirror 1574.
With reference to
Left mirror 1576 receives light beams 1588 and 1590, which include information respective of the left side view image of three-dimensional object 1582. Since the non-reflective side of rotating mirror 1578 is facing left mirror 1576, this non-reflective side absorbs light beams 1588 and 1590. Thus, light beams 1588 and 1590 reach neither optical assembly 1564 nor light sensor array 1566, nor is reflected or refracted light incident upon the three-dimensional object 1582, and light sensor array 1566 does not detect the left side view image of three-dimensional object 1582.
With reference to
Since the non-reflective side of rotating mirror 1578 faces right mirror 1574, this non-reflective side absorbs light beams 1584 and 1586. Thus, light beams 1584 and 1586 reach neither optical assembly 1564 nor light sensor array 1566, nor is reflected or refracted light incident upon the three-dimensional object 1582, and light sensor array 1566 does not detect the right side view image of three-dimensional object 1582. Rotating mirror 1578, then rotates 90 degrees clockwise to the position illustrated in
Alternatively, an optical element, such as an optical diaphragm, prism, mirror and the like, replaces rotating mirror 1578. Half of the optical diaphragm is transparent and the other half is opaque. The optical diaphragm oscillates about an axis, by an electronic element, such as piezoelectric element, and the like, such that the transparent and the opaque portions of the diaphragm are alternately located above right mirror 1574 and left mirror 1576.
According to another aspect of the disclosed technique, two fiberscopes are employed whose inlets are substantially spaced apart relative to the outlets thereof. One fiberscope obtains a right side view image of the three-dimensional object, while the other fiberscope obtains a left side view image of the three-dimensional object.
Reference is now made to
A fiberscope is a flexible longitudinal element, which is generally employed for obtaining an image of an object which is obstructed by other objects and can not be viewed directly. The fiberscope includes a substantially large number of fibers. One end of each fiber receives the image of a substantially small portion of the object at the inlet of the fiberscope and conveys this image to the other end of the same fiber, at the outlet of the fiberscope. Thus, the plurality of the fibers, together provide a complete image of the object at the outlet of the fiberscope, duplicating the image detected by the fiberscope at the inlet thereof.
The relative positions of the ends of the fibers at the outlet of the fiberscope, are the same as the relative positions of the fibers at the inlet of the fiberscope (i.e., substantially no twist of the fibers along the length of the fiberscope is allowed). Otherwise, the image of the object at the outlet of the fiberscope will be skewed and different from the image of the object as viewed by the inlet of the fiberscope.
Right fiberscope 1636 includes an image inlet 1640, an image outlet 1642 and a plurality of fibers 1644. Left fiberscope 1638 includes an image inlet 1646, an image outlet 1648 and a plurality of fibers 1650. Multiple aperture 1624 includes a right aperture 1652R and a left aperture 1652L. Multiple aperture 1624 is similar to multiple aperture 804, as described herein above in connection with
Light directing assembly 1622 is located between a three-dimensional object 1654 and multiple aperture 1624. Multiple aperture 1624 is located between light directing assembly 1622 and optical assembly 1626. Optical assembly 1626 is located between multiple aperture 1624 and light sensor array 1628. Inlet lenses 1666 and 1668 are located between three-dimensional object 1654 and image inlets 1640 and 1646, respectively. Outlet lenses 1670 and 1672 are located between multiple aperture 1624 and image outlets 1642 and 1648, respectively.
Right fiberscope 1636 and left fiberscope 1638 are bent, such that image inlets 1640 and 1646 are spaced apart and image inlets 1642 and 1648 are located close together. In this manner, right fiberscope 1636 obtains an image of three-dimensional object 1654 from the right side thereof, which is substantially different from another image obtained by left fiberscope 1638, from the left side of three-dimensional object 1654.
Light beams 1658 and 1660 include information respective of the right side view image of three-dimensional object 1654. Inlet lens 1666 focuses light beams 1658 and 1660 on image inlet 1640. Fibers 1644 convey light beams 1658 and 1660 to image outlet 1642. Outlet lens 1670 focuses light beams 1658 and 1660 on right aperture 1652R. Since right aperture 1652R is open, light beams 1658 and 1660 reach optical assembly 1626R, optical assembly 1626R focuses light beams 1658 and 1660 on light sensor array 1628. Controller 1630 enables light sensor array 1628 to detect a right side view image of three-dimensional object 1654, according to the state of multiple aperture 1624 (i.e., when right aperture 1652R is open). Controller 1630 stores this right side view image in storage unit 1632.
Light beams 1662 and 1664 include information respective of the left side view image of three-dimensional object 1654. Inlet lens 1668 focuses light beams 1662 and 1664 on image inlet 1646. Fibers 1650 convey light beams 1662 and 1664 to image outlet 1648. Outlet lens 1672 focuses light beams 1662 and 1664 on left aperture 1652L. Since left aperture 1652L is closed, light beams 1662 and 1664 are blocked and light sensor array 1628 does not detect the left side view image of three-dimensional object 1654.
In another mode of apparatus 1620 (not shown), right aperture 1652R is closed and left aperture 1652L is open. Thus, left aperture 1652L allows light beams 1662 and 1664 to pass there through and reach optical assembly 1626. Optical assembly 1626 focuses light beams 1662 and 1664 on light sensor array 1628. Controller 1630 enables light sensor array 1628 to detect a left side view image of three-dimensional object 1654, according to the state of multiple aperture 1624 (i.e., when left aperture 1652L is open). Controller 1630 stores this left side view image in storage unit 1632. Image processor 1634 produces a video signal for a stereoscopic display, such as stereoscopic display 214 (
According to another aspect of the disclosed technique, a plurality of an arm of Y-junction fibers are spaced from a plurality of another arm of the Y-junction fibers. The plurality of each arm of the Y-junction fibers alternately transfer an image of a three-dimensional object, as viewed from the respective side, to the plurality of the legs of the Y-junction fibers.
Reference is now made to
Apparatus 1700 includes an image differentiator 1702, a right lens 1748, a left lens 1750, a light directing assembly 1704, an optical assembly 1706 and a light sensor array 1708. Image differentiator 1702 can include a right side filter 1710 and a left side filter 1712, similar to right side filter 1202 and left side filter 1204, respectively, as described herein above in connection with
If image differentiator 1702 is a filter type image differentiator, then image differentiator 1702 includes right side filter 1710 and left side filter 1712. In this case, apparatus 1700 further includes two illuminators (not shown) similar to illuminators 1212 and 1214 as described herein above in connection with
Light directing assembly 1704 includes a sleeve 1714, a right inlet 1716, a left inlet 1718, an outlet 1720 and a plurality of split fibers 1722. Sleeve 1714 includes a right section 1724, a left section 1726 and a common section 1728.
Image differentiator 1702 is located between a three-dimensional object 1730, and right lens 1748 and left lens 1750. Right lens 1748 is located in front of right inlet 1716 and it produces a right side view image of three-dimensional object 1730 on right inlet 1716. Left lens 1750 is located in front of left inlet 1718 and it produces a left side view image of three-dimensional object 1730 on left inlet 1718. Light directing assembly 1704 is located between right lens 1748 and left lens 1750, on the one side, and optical assembly 1706, on the other side. Optical assembly 1706 is located between light directing assembly 1704 and light sensor array 1708.
With reference to
Right inlet 1716 can receive a right side view image of three-dimensional object 1730 and left inlet 1718 can receive a left side view image thereof. The controller controls the operation of image differentiator 1702 and the two illuminators, such that right inlet 1716 and left inlet 1718 alternately receive the right side view image and the left side view image, respectively, of three-dimensional object 1730.
Each of a plurality of the right arms 1732 receives a substantially small portion of the right side view image of three-dimensional object 1730 and transfers this portion of the image to the respective common arm 1736. The plurality of the common arms 1736, together produce the complete right side view image of three-dimensional object 1730, as received by the plurality of the right arms 1732. In the same manner, a plurality of left arms 1734 transfers the left side view image of three-dimensional object 1730, to the plurality of common arms 1736. The common arms 1736 together produce the complete left side view image of three-dimensional object 1730, as received by the plurality of the left arms 1734.
The relative positions of common arms 1736 of split fibers 1722 within common section 1728, are substantially the same as the relative positions of right arms 1732 within right section 1724, and the relative positions of left arms 1734 within left section 1726. Otherwise, the image of three-dimensional object 1730 at outlet 1720 will be skewed and different from the image of three-dimensional object 1730 as viewed by either right inlet 1716 or left inlet 1718.
If the split fibers 1722 are placed within sleeve 1714, such that junctions 1742 (
In the example set forth in
Light beams 1744 and 1746, which include information respective of the left side view image of three-dimensional object 1730, reach left side filter 1712. Since left side filter 1712 is not operative, light beams 1744 and 1746 are blocked and do not reach light sensor array 1708.
In another mode of apparatus 1700 (not shown), right side filter 1710 blocks light beams 1738 and 1740, while left side filter 1712 lets through the light beams 1744 and 1746. Left lens 1750 focuses light beams 1744 and 1746 on left inlet 1718, wherein left lens 1750 images the points on three-dimensional object 1730 from which light beams 1744 and 1746 have arrived, on left inlet 1718. In this case, the plurality of left arms 1734 transfer light beams 1744 and 1746 to outlet 1720, via the respective plurality of common arms 1736. Optical assembly 1706 receives light beams 1744 and 1746 from outlet 1720 and optical assembly 1706 focuses light beams 1744 and 1746 on light sensor array 1708. The processor enables light sensor array 1708 to detect a left side view image of three-dimensional object 1730, according to the state of image differentiator 1702 (i.e., when left side filter 1712 is open).
Reference is now made to
Apparatus 1800 includes a right side filter 1802, a left side filter 1804, a periscope assembly 1806, an optical assembly 1808, a duo-tone rotating disk 1810, a light sensor array 1812, an illuminator 1814, a controller 1816, a storage unit 1818 and an image processor 1820. Right side filter 1802 is a light filter, which admits light in only a predetermined range of wavelengths. Left side filter 1804 is a light filter which admits light in another predetermined range of wavelengths, different than the range of wavelengths which is set for right side filter 1802. Periscope assembly 1806 is similar to periscope assembly 1206, as described herein above in connection with
Illuminator 1814 provides light in at least the range of wavelengths defined by filtering portions 1822 and 1824. In the example set forth in
Right side filter 1802 and left side filter 1804 are located between a three-dimensional object 1826 and periscope assembly 1806. Periscope assembly 1806 is located between right side filter 1802 and left side filter 1804, and optical assembly 1808. Optical assembly 1808 is located between periscope assembly 1806 and duo-tone rotating disk 1810. Duo-tone rotating disk 1810 is located between optical assembly 1808 and light sensor array 1812. Duo-tone rotating disk 1810, light sensor array 1812, controller 1816, storage unit 1818 and image processor 1820 are interconnected via a bus 1848.
With reference to
Optical assembly 1808 receives light beams 1832, 1834, 1840 and 1842 at inlets thereof (not shown), and directs light beams 1832, 1834, 1840 and 1842 from an outlet thereof (not shown) to duo-tone rotating disk 1810. In the example set forth in
Controller 1816 enables light sensor array 1812 to detect a right side view image of three-dimensional object 1826, according to the position of duo-tone rotating disk 1810 relative to light sensor array 1812 (i.e., when filtering portion 1822 is located above light sensor array 1812). Controller 1816 stores this right side view image in storage unit 1818.
With reference to
Reference is now made to
With reference to
With reference to
Reference is now made to
With reference to
With reference to
According to another aspect of the disclosed technique, the two-dimensional light sensor array is replaced by a one-dimensional light sensor array and a rotating mirror, which swivels about an axis perpendicular to the stereoscopic axis. The rotating mirror rotates about an axis which is parallel to the one-dimensional light sensor array, thereby continuously scanning the surface of a three-dimensional body. The rotating mirror directs the scanned image to the one-dimensional light sensor array, via an image differentiator, a light directing assembly and an optical assembly. A controller coupled with the one-dimensional light sensor array enables the one-dimensional light sensor array to detect images of different regions of the three-dimensional object in sequence. The image differentiator differentiates between a line of the right side view image and a line of the left side view image of each of these different regions, before these lines of image reach the one-dimensional light sensor array.
Reference is now made to
With reference to
Static polarizers 2012 and 2014 are located between scanning element 2002 and image directing assembly 2006. Image directing assembly 2006 is located between static polarizers 2012 and 2014 on one side and dynamic polarizer 2016 on the other side. Dynamic polarizer 2016 is located between image directing assembly 2006 and optical assembly 2008. Optical assembly 2008 is located between dynamic polarizer 2016 and image detector 2010.
With further reference to
Scanning element 2002 being at a certain angular position, directs an image line of a region 2024 of three-dimensional object 2022, to static polarizers 2012 and 2014. Right periscopic prism 2018 receives a line of the right side view image of region 2024 via static polarizer 2012 and left periscopic prism 2020 receives a line of the left side view image of region 2024 via static polarizer 2014. Right periscopic prism 2018 and left periscopic prism 2020 direct the line of the right side view image and the line of the left side view image of region 2024 to dynamic polarizer 2016. In the example set forth in
With further reference to
Hence, the light beams which define the line of the right side view image of region 2026 pass through dynamic polarizer 2016 and reach one-dimensional light sensor array 2010, while the light beams which define the line of the left side view image of region 2026 are blocked by dynamic polarizer 2016 and do not reach one-dimensional light sensor array 2010. A controller which is coupled with scanning element 2002 and to one-dimensional light sensor array 2010, enables one-dimensional light sensor array 2010 to detect a line of an image of three-dimensional object 2022, according to the angular position of scanning element 2002. It is noted that scanning element 2002 can either rotate continuously, or rotate back and forth between two angular positions.
Alternatively, the image detector is a two-dimensional light sensor array operating in time delay integration (TDI) mode. The scanning element scans a plurality of successive two-dimensional regions of the three-dimensional object. The scanning element directs the two-dimensional images of these two-dimensional regions, in succession, to the image detector. A controller is coupled with the scanning element and to the image detector. The controller successively shifts the electronic charges from one row of the image detector to the other row in turn, along the columns of the image detector in synchrony with the scanning movement of the scanning element. After shifting the electronic charges from a first row to a second row, the controller resets the first row. In this manner, the sum of the electronic charges of all the rows are accumulated in the last row of the two-dimensional light sensor array. The controller delivers the charges from the last row of the image detector, in sequence and in synchrony with the scanning movement of the scanning element, to an image processor. The image processor produces a substantially sharp stereoscopic image of the region of the three-dimensional object, which the scanning element repeatedly scans.
It is noted, that if the image detector does not operate in TDI mode (i.e., the controller does not shift the charges from one column to the other), then the image processor produces a blurred stereoscopic image of the three-dimensional object. This is so, because the scanning element provides images of successive regions of the three-dimensional object to the image detector. The image processor produces a stereoscopic image of the three-dimensional object and the stereoscopic image is blurred according to the scanning speed of the scanning element.
According to another aspect of the disclosed technique, a right side filter and a left side filter are employed, each admitting an image at two different ranges of wavelengths. When the three-dimensional body is sequentially illuminated with light at each of the first ranges of wavelengths, the right side filter sequentially directs a right side view image of the three-dimensional object to the image detector, at each one of the first ranges of wavelengths. Likewise, when the three-dimensional body is sequentially illuminated at each of the second ranges of wavelengths, the left side filter sequentially directs a left side view image of the three-dimensional object to the image detector, at each one of the second ranges of wavelengths.
Reference is now made to
Apparatus 2040 includes a right side filter 2042, a left side filter 2044, an image detector 2046, an illuminator 2048, a controller 2050, a storage unit 2052 and an image processor 2054. Right side filter 2042 and left side filter 2044 are located between a three-dimensional object 2056 and image detector 2046. Controller 2050 is coupled with illuminator 2048. Image detector 2046 controller 2050, storage unit 2052 and image processor 2054 are coupled together via a bus 2058.
Right side filter 2042 admits light within the ranges of wavelengths ΔR1, ΔG1 and ΔB1. Left side filter 2044 admits light within the ranges of wavelengths ΔR2, ΔG2 and ΔB2. Illuminator 2048 sequentially emits light at each of the ranges of wavelengths ΔR1, ΔG1, ΔB1, ΔR2, ΔG2 and ΔB2.
With reference to
With reference to
Alternatively, illuminator 2048 is replaced by a sequential multi-wavelength illuminator which emits light at a mixture of the ranges of wavelengths ΔR1, ΔG1 and ΔB1 and at a mixture of the ranges of wavelengths ΔR2, ΔG2 and ΔB2. The sequential multi-wavelength illuminator sequentially emits light at each of the mixtures of the ranges of wavelengths ΔR1, ΔG1 and ΔB1, and at each of the mixtures of the ranges of wavelengths ΔR2, ΔG2 and ΔB2. When the sequential multi-wavelength illuminator emits light at the mixture of the ranges of wavelengths ΔR1, ΔG1 and ΔB1, right side filter 2042 directs a full color right side view image of three-dimensional object 2056, at the mixture of the ranges of wavelengths ΔR1, ΔG1 and ΔB1, to image detector 2046. When the sequential multi-wavelength illuminator emits light at the mixture of the ranges of wavelengths ΔR2, ΔG2 and ΔB2, left side filter 2044 directs a full color left side view image of three-dimensional object 2056, at the mixture of the ranges of wavelengths ΔR2, ΔG2 and ΔB2, to image detector 2046.
Further alternatively, illuminator 2048 is replaced by a multi-wavelength illuminator which emits light at a range of wavelengths which encompasses the ranges of wavelengths ΔR1, ΔG1, ΔB1, ΔR2, ΔG2 and ΔB2 and a duo-tone rotating disk in located between the right side filter and the left side filter at one side and the image detector at the other. The duo-tone rotating disk is divided to two transparent portions. One transparent portion of the duo-tone rotating disk admits light at the ranges of wavelengths ΔR1, ΔG1 and ΔB1, and the other transparent portion thereof, admits light at the ranges of wavelengths ΔR2, ΔG2 and ΔB2. The multi-wavelength illuminator continuously illuminates the three-dimensional object. As the duo-tone rotating disk rotates, the right side filter and the left side filter sequentially direct a full color right side view image and a full color left side view image, respectively, of the three-dimensional object, to the image detector.
Alternatively, right side filter 2042 and left side filter 2044 are spaced apart. In this case right side filter 2042 receives a right side view image of three-dimensional object 2056, which is considerably more distinct than a left side view image thereof, thereby allowing image processor 2054 to produce a more realistic full color stereoscopic image of three-dimensional object 2056. It is noted that instead of the duo-tone rotating disk, other types of rotating disks can be employed, such as a multi-wavelength rotating disk (
Reference is now made to
In procedure 2082, a first set of differentiating wavelengths which is included in the first ranges of filter wavelengths and excluded from the second ranges of filter wavelengths, is determined. With reference to
In procedure 2086, an object is sequentially illuminated with the first set of differentiating wavelengths and with the second set of differentiating wavelengths. With reference to
In procedure 2088, a first image is detected when the first set of differentiating wavelengths is present and a second image is detected when the second set of differentiating wavelengths is present. With reference to
According to another embodiment, differentiation is performed by sequentially admitting light at the different sets of wavelengths, by a sequential filtering device, such as a rotating disk, an alternating filter, and the like. According to this embodiment, procedure 2090 replaces procedure 2086. In procedure 2090, light is admitted sequentially at the first set of differentiating wavelengths and at the second set of differentiating wavelengths.
The light differentiator can be any optical device which can differentiate between different wavelengths (e.g., by means of illumination, reflection or filtration). For example, the light differentiator can be a rotating disk divided into filtering sectors, wherein each filtering sector filters light at wavelengths which are included in one of the right side filter and the left side filter and excluded from the other of these two filters. Alternatively, a reflective rotating disk can be employed, which is divided into a plurality of reflecting sectors, where each reflecting sector reflects light at a different wavelength. Further alternatively, a multi-state flipping filter can be employed, which is mechanically flipped from one light filter to the other, in sequence. Other types of sequential filters, such as those which are operated electrically rather than mechanically, are applicable to this embodiment. Alternatively, the light differentiator can be a set of partially reflective mirrors that can be operated sequentially, each reflecting light at wavelengths which are included in one of the right side filter and the left side filter and excluded from the other of these two filters (e.g., a partially reflective mirror which reflects light at CYMG1 and another partially reflective mirror which reflects light at CYMG2).
Reference is further made to
With reference to
In examples described above, the light differentiator differentiates between two groups of wavelengths, where each group of wavelengths includes three wavelengths (i.e., R, G and B). Thus, the light differentiator of the stereoscopic imaging apparatus differentiates between two red wavelengths (R1 and R2), two green wavelengths (G1 and G2) and two blue wavelengths (B1 and B2). As noted above, the light differentiator can be for example, an illuminator, a light filtering element or a light reflecting element.
However, it is noted that each of the two groups of wavelengths can include more than three wavelengths and for that matter, any number of wavelengths. For example, high quality spectrometers are capable to split the light to 20 or 40 or more different wavelengths (e.g., IR1, IR2, IR3, IR4, . . . IRn, R1, R2, R3, . . . , Rm, G1, G2, G3, . . . , Gp, B1, B2, B3, . . . , Bq, UV1, UV2, UV3, . . . , UVs, and the like).
Reference is now made to
With reference to
Periscope assembly 2142 is located between a three-dimensional object 2166 and image differentiator 2144. Image differentiator 2144 is located between periscope assembly 2142 and optical assembly 2146. Optical assembly 2146 is located between image differentiator 2144 and light sensor array 2148.
The X axis designates the longitudinal axis of apparatus 2140. The X axis together with the Y and Z axes, form a rectilinear coordinate system. In the following description, the right hand rule applies to this coordinate system. For example, the phrase “a tilt of positive 45 degrees about the Z axis”, means a tilt of 45 degrees about the Z axis in the direction of the fingers, when the thumb points in the direction of the Z axis. On the other hand, the phrase “a tilt of negative 45 degrees about the Z axis”, means a tilt of 45 degrees about the Z axis in the direction of the fingers, when the thumb points in a direction opposite to the Z axis.
The reflecting surface of right front mirror 2150 is tilted by preferably positive 45 degrees about the Y axis from the Z-Y plane and by preferably negative 30 degrees about the Z axis, from the Z-X plane. The reflecting surface of left front mirror 2152 is tilted by preferably positive 45 degrees about the Y axis from the X-Y plane and by preferably negative 30 degrees about the Z axis, from the Z-X plane.
The reflecting surface of right middle mirror 2154 is tilted by preferably negative 45 degrees about the X axis from the Z-X plane and by preferably negative 30 degrees about the Z axis, from the Z-X plane. The reflecting surface of left middle mirror 2156 is tilted by preferably positive 45 degrees about the X axis from the Z-X plane and by preferably negative 30 degrees about the Z axis, from the Z-X plane.
The reflecting surfaces of right rear mirror 2158 and left rear mirror 2160 are tilted by preferably negative 60 degrees about the Z axis from the Z-X plane. Hence, periscope assembly 2142 is tilted preferably by negative 30 degrees about the Z axis from the Z-X plane.
Right front mirror 2150 receives a light beam 2168 respective of a right side view image of three-dimensional object 2166. Since periscope assembly 2142 is tilted by substantially negative 30 degrees about the Z axis, light beam 2168 is located on a plane which is tilted by substantially negative 30 degrees from the Z-X plane, about the Z axis. Right front mirror 2150 directs a reflection of light beam 2168 toward right middle mirror 2154, as a light beam 2170. Light beam 2170 is located on the Z-X plane.
Right middle mirror 2154 directs a reflection of light beam 2170 toward right rear mirror 2158, as a light beam 2172. Light beam 2172 is located at the intersection of the X-Y plane and a plane which is tilted about the Z axis by approximately positive 60 degrees from the Z-X plane. Right rear mirror 2158 directs a reflection of light beam 2172 toward image differentiator 2144, as a light beam 2174. Light beam 2174 points in a direction substantially parallel to the X axis. In the example set forth in
With reference to
Referring back to
It is noted that right front mirror 2150, right middle mirror 2154 and right rear mirror 2158 can be incorporated in a right prism, wherein the right prism is titled sideways relative to the longitudinal axis of the apparatus. In this case, each of the right front mirror 2150, right middle mirror 2154 and right rear mirror 2158 represents the respective reflective surface of the right prism. Likewise, right front mirror 2152, right middle mirror 2156 and right rear mirror 2160 can be incorporated in a left prism, wherein the left prism is titled sideways relative to the longitudinal axis of the apparatus, by the same amount as the right prism. Thus, the right prism receives a right side view image of a three-dimensional object which is located at a side of the apparatus, while the left prism receives a left side view image of the three-dimensional.
It is noted that above optical structure provides a clear, straight and undistorted image at each of the right and left channels.
Reference is now made to
With reference to
Objective lens assembly 2204 generates two overlapping images on the image plane (i.e., on the plane defined by the light sensor array 2208). One of these images arrives from left pupil PL and the other image arrives from right pupil PR. With reference to
Similarly, referring to
It is noted that in the present example, objective lens assembly 2204 is telecentric. Accordingly, light beams 2270B, 2272B and 2274B are parallel there between, as are light beams 2220B, 2222B and 2224B. Hence, each lenticular lens receives light beams at one of two specific directions, and directs these light beams to one of two specific light sensors. Alternatively, the objective lens assembly may be nearly telecentric, in which case these light beams are only approximately parallel, but the lenticular lens still separates between the two groups of light beams. In general, the objective lens assembly should direct the light beams from the left pupil in a direction from a first set of directions, and the light beams from the right pupil in a direction from a second set of directions.
According to the present embodiment, the pupils PL and PR define the “eyes” of the optical device, which are required for stereoscopic vision. It is noted that the light beams arrive at the lenticular elements substantially in one of two specific directions. Hence, each lenticular element distinguishes precisely between the light received from the left pupil and that received from the right pupil.
Alternatively, the aperture stop includes “soft” pupils, instead of the pupils PL and PR. Reference is now made to
Further alternatively, the left and right pupils may be “virtual pupils”. Accordingly, the plane of aperture stop 2210 (
It will be appreciated by persons skilled in the art that the disclosed technique is not limited to what has been particularly shown and described here in above. Rather the scope of the disclosed technique is defined only by the claims which follow.
This application is a divisional of Ser. No. 10/145,418, filed May 13, 2002, which is a continuation-in-part of application Ser. No. 09/785,791, filed on Feb. 16, 2001 and 09/785,512, filed on Feb. 16, 2001, which are continuation-in-parts of application Ser. No. 09/699,624, filed on Oct. 30, 2001, which is a continuation-in-part of application Ser. No. 09/257,850, filed on Feb. 25, 1999 and which application(s) are incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 2639653 | Fischer | May 1953 | A |
| 3932699 | Tripp | Jan 1976 | A |
| 4414470 | Nakaoka | Nov 1983 | A |
| 4437764 | Levine et al. | Mar 1984 | A |
| 4873572 | Miyazaki et al. | Oct 1989 | A |
| 4945407 | Winnek | Jul 1990 | A |
| 4959641 | Bass et al. | Sep 1990 | A |
| 5034805 | Ishizaka | Jul 1991 | A |
| 5076687 | Adelson | Dec 1991 | A |
| 5121452 | Stowe et al. | Jun 1992 | A |
| 5192969 | Igarashi et al. | Mar 1993 | A |
| 5233416 | Inoue | Aug 1993 | A |
| 5428386 | D'Alfonso et al. | Jun 1995 | A |
| 5490015 | Umeyama | Feb 1996 | A |
| 5527263 | Zobel et al. | Jun 1996 | A |
| 5552840 | Ishii et al. | Sep 1996 | A |
| 5588948 | Takahashi et al. | Dec 1996 | A |
| 5594497 | Ahem et al. | Jan 1997 | A |
| 5603687 | Hori et al. | Feb 1997 | A |
| 5604531 | Iddan et al. | Feb 1997 | A |
| 5606436 | Shapiro | Feb 1997 | A |
| 5606455 | Eichenlaub | Feb 1997 | A |
| 5613936 | Czarnek et al. | Mar 1997 | A |
| 5653677 | Okada | Aug 1997 | A |
| 5743846 | Takahashi et al. | Apr 1998 | A |
| 5743847 | Nakamura et al. | Apr 1998 | A |
| 5751341 | Chaleki et al. | May 1998 | A |
| 5760827 | Faris | Jun 1998 | A |
| 5776049 | Takahashi | Jul 1998 | A |
| 5800341 | McKenna | Sep 1998 | A |
| 5812187 | Watanabe | Sep 1998 | A |
| 5825534 | Strahle | Oct 1998 | A |
| 5828487 | Greening et al. | Oct 1998 | A |
| 5835194 | Morton | Nov 1998 | A |
| 5865829 | Kitajima | Feb 1999 | A |
| 5868664 | Speier et al. | Feb 1999 | A |
| 5966168 | Miyazaki | Oct 1999 | A |
| 5991074 | Nose et al. | Nov 1999 | A |
| 6075555 | Street | Jun 2000 | A |
| 6306082 | Takahashi et al. | Oct 2001 | B1 |
| 6593957 | Christie | Jul 2003 | B1 |
| Number | Date | Country |
|---|---|---|
| 2157073 | Jun 1996 | CA |
| WO 0022975 | Oct 1999 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20080158344 A1 | Jul 2008 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10145418 | May 2002 | US |
| Child | 11932502 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 09785791 | Feb 2001 | US |
| Child | 10145418 | US | |
| Parent | 09785512 | Feb 2001 | US |
| Child | 09785791 | US | |
| Parent | 09699624 | Oct 2000 | US |
| Child | 09785791 | US | |
| Parent | 09527850 | Feb 1999 | US |
| Child | 09699624 | US |
