Patent Application
20110056290
1. Field of the Invention
The present invention relates to machine vision systems and, more specifically, to a system and method for automatically identifying the level of liquids in a vessel.
2. Description of the Related Art
Machine vision plays an important role in automated and robotic systems, such as assembly line manufacturing, quality control inspection, and sample processing. Conventional systems are generally comprised of an optical imager, such as a charged coupled device (CCD) or similar device using digital imaging technology, which is positioned to capture images of objects that pass in front of the imager. In low-light or enclosed applications, machine vision systems may include an illumination source, such as a bank of light emitting diodes (LEDs), positioned proximately to the imager. The images are subsequently processed to decode information contained in the resulting two-dimensional image, such as 1D linear codes, 2D stacked/matrix codes, OCR fonts, and postal codes. The image captured by the machine vision system may also be subjected to more advanced processing, such as shape recognition or detection algorithms, that provide information about the object of interest in the image.
In robotic sample handling systems, such as blood analyzers and the like, samples are moved to and from diagnostic modules for automatic testing and retesting using a loading rack that holds a plurality of carriers, such as test tubes filled with samples. The samples are generally contained within vessels, such as test tubes, that are accessed by the handling system for testing procedures. For example, a pipette may be lowered into a test tube and used to withdraw a predetermined amount of fluid from the test tube for further processing. In order to perform a proper withdrawal, the robotic handling system must be able to determine the appropriate distance into the vessel into which the pipette must be inserted in order to position the tip of the needle in the appropriate portion of the sample. For example, a test tube containing a blood sample that has been treated and centrifuged may contain multiple, individual layers of liquids that result from the separation of the blood and treatment materials. The proper location of the pipette may therefore require determining the location of the various layers so that the pipette may be inserted the appropriate distance into the vessel so that material from any of the layers may be withdrawn for further processing.
Current methods for determining liquid levels are relatively ineffective and do not provide accurate information. For example, capacitive approaches cannot determine the type of cap placed on the vessel, do not provide any information about sample quality, and will not work if the vessel contains a layer of foam. In addition, capacitive approaches have low reliability with respect to the detection of the location of the level. Ultrasonic approaches are also unable to detect the type of cap, will not work if there is a cap or if there are multiple layer levels, do not provide any information about sample quality, will not work with foam, and are only moderately effective. Finally, pressure based systems are also unable to determine the type of cap, will not work if a cap is present, cannot detect multiple layers, cannot provide any information about sample quality, will not work if foam is present, and are only moderately accurate. All of these approaches also require contact with the vessel or very close proximity to the vessel for operation.
The present invention provides a system and method for identifying the levels of one or more liquids in a vessel using one ore more appropriately positioned light sources and an optical imager for capturing digital images of the illuminated vessel. For example, a line generator may be positioned on one side of the vessel and oriented to project a line of light that intersects each layer of liquid in the vessel. An optical imager is positioned on the opposing side of the vessel from the laser line generator and oriented to capture an image of the vessel along with whatever portions of the laser line are visible through the vessel. A second light source, such as an LED, may be positioned above or below the vessel and oriented to project light downwardly or upwardly, respectively, through the vessel and its contents. Captured images of the vessel are then processed by a programmable device, such as a microcontroller, to determine the levels of materials in the vessel based on the optical characteristics revealed in the captured image.
The present invention will be more fully understood and appreciated by reading the following Detailed Description in conjunction with the accompanying drawings, in which:
Referring now to the drawings, wherein like reference numerals refer to like parts throughout, there is seen in
As further seen in
When one or more labels are present, the amount of light coupled into tube 12 is significantly reduced and is difficult for a vision-based system to detect and measure the light exiting tube 12. For this reason, a non-imaging photodiode-based optical system as seen in
As seen in
Imager 22 is preferably positioned oppositely from laser line generator 28, but can be positioned in any location that allows imager 22 to capture an image of the projected laser line 26. It should be recognized by those of skill in the art that the particular location of imager 22, laser line generator 18 and LED 24 may vary relative to rack 20, and each other, depending on the particular structure of the handling system, provided that the alignment of imager 22, laser line generator 18 and LED 24 with respect to each other allows imager 22 to capture images of test tube 12 that include the projected line 26 of the laser line after it passes through tube 12.
Referring to
In a preferred embodiment of the present invention, laser line generator 18 and upwardly projecting LED 24 are also interconnected to imager 22 so that imager 22 can control the timing and duration of laser line generation and test tube illumination. Alternatively, laser line generator 18 and LED 24 may be powered and controlled by another device. Imager 22 further includes a host interface 34 for communicating with a host device, such as the robotic handling system. Interface 34 may also comprise a conventional RS232 transceiver and associated 12 pin RJ style jack or other conventional buses, such as USB, IEEE 1394, I2C, SPI, or PCMCIA, or other connector styles, such as an FFC style. Imager 22 is preferably connected to a host system through either RS232 serial port or TCP/IP protocol and configured as a slave device to accept control and configuration commands from and returning the information to the host system.
Microcontroller 32 is preferably programmed to process captured images locally, thereby avoiding the need to transmit captured images to a host and improving the speed and efficiency of system 10. Preferably, microcontroller is programmed to interpret captured images of tube 12 to determine the location of the levels of any liquids contained in tube 12. The level information may then be transmitted to a host, such as robotic handler 14, so that handler 14 can more accurately position a pipette 36 the proper distance into tube 12 for extraction of any of the various liquids contained in tube 12 with withdrawing any liquid from adjoining levels.
More specifically, microcontroller 32 may be programmed to interpret images captured by imager 22 to determine the level of each liquid inside test tube 12. In addition, microcontroller 32 may be programmed to further determine whether test tube 12 has a cap positioned thereon and, if so, determine what type of cap. Microcontroller 32 may further be programmed to provide information about the quality of the sample and its color indices, or to decode any information placed onto test tube 12, such as a barcode.
With respect to liquid level detection, the present invention employs a line of light or laser line that exploits the differences in the indices refraction between the layers of liquids and liquid and art contained in the tube singularly or in combination with a strong upward or, optionally, downward directed LED or laser spot illumination that highlights the circumference at the top liquid/air interface or at subsequent liquid/liquid interfaces. As seen in
Referring to
With respect to test tube and test tube cap identification, application Ser. No. 12/145,619, hereby incorporated by reference in its entirety, details a system and method for determining the presence of a cap on a test tube as well as the type of test tube and cap, using images captured by an optical imager such as imager 22.
With respect to barcode decoding, conventional imager that may serve as all or part of imager 22 discussed above may be equipped with algorithms for decoding all the symbologies shown in Table 1 below.
Barcode symbologies and their decoding algorithms follow the international barcode standards and those of skill in the art will instantly appreciate how to accomplish this functionality through the use of off-the-shelf imagers or decoding packages.
Imager 22 may be programmed to capture a series of images of test tube 22 that are stitched together to form a panoramic image of test tube 22. Image stitching involves combining multiple photographic images with overlapping fields of view to produce a segmented panorama or high-resolution image. Those of skill in the art will recognize that a panoramic image may be processed according to the present invention to determine the levels of liquids in test tube 12 rather than using a single captured image.
Most approaches to image stitching require nearly exact overlaps between images and identical exposures to produce seamless results. This process is also known as mosaicing. The stitching process involves three stages: image calibration, image registration, and image blending. Image calibration consists of processing to improve image quality and ease of stitching, such as perspective correction, vignetting correction, and chromatic aberration correction. Image registration involves analysis for translation, rotation, and focal length. Alternatively, direct or feature-based image alignment methods may be used. Direct alignment methods search for image orientations that minimize the sum of absolute differences between overlapping pixels. Feature-based methods determine proper image orientations by identifying features that appear in multiple images and overlapping them. Finally, image blending involves combining of the images and may further entail color correction to match the adjoining areas of the component images for color, contrast, and brightness to avoid visibility of the seams, dynamic range extension, and motion compensation, such as deghosting and deblurring, to compensate for moving objects.
Generally, image stitching is a very computation-intensive process, and usually not suitable to be performed on-line on imagers 22 which are often limited by both speed and memory. However, by shifting some of the computation to off-line calibration, operational image stitching computations can be greatly improved. Off-line calibration is grounded in the fact that imager 22 is fixed mounted, so the relative position between the placement of tubes 12 and imager 22 is fixed. Thus, the perspective transformation from the imager of tube 12 to imager 2 is almost fixed. Because the sample tube is held still upright by robotic handler 14, which must be able to rotate tube 12 and conventional robotic systems can in fact do so, the geometrical perspective shift parameters from one image to the next will be limited to the horizontal direction. The variation for the other geometrical parameters, such as vertical direction translation shift, scale, and rotation shift are small or negligible, thereby greatly simplifying the image registration process by limiting the search space. If tube rotation speeds are kept constant by robotic handler 14, the time difference between subsequent image captures should be relatively constant. The optimal adjustment between imager capture speed and image rotation speed can provide a nearly exact overlap between the consecutive tube images and the shift parameters should be nearly identical during the stitching process. In system 10, imager 14 may also be configured to use identical exposures when capturing images to produce more seamless results. Finally, as the stitched images are used mainly for barcode decoding and liquid level detection, not for viewing, a small boundary effect may be tolerated. Therefore, stitching can be further simplified, such as by avoiding the blending, color correction, chromatic correction required for images that are to be viewed visually. Based on these characteristics of system 10, most of the image stitching computations, i.e., the initial estimation of image stitching geometry shift parameters, may be transferred to off-line processing and be pre-computed by host 16.
Following is an example stitching process, although those of ordinary skill in the art will recognize that other methods may be used. Let x′=[x′,y′,1] and x=[x,y,1] denote the corresponding position between the current image and previous image. The most general planar 2D transform is the eight-parameter perspective transform like
The image registration is just seeking to estimate H using the corresponding matching image points from the consecutive two images. During off-line processing, it is possible to use a calibration image to get an estimated transform
x′=Hx=(
Reviewing the characteristics described above, ΔH should be very small. As noted above, a search for ΔH can be limited to the horizontal direction, i.e., Δh02 within a lot of pixels. The off-line estimation of the transform matrix
To further reduce the computation for stitching process, the requirement for tube stitching reconstruction fidelity may be changed from the preferred stitching process. As explained above, the tube rotation images are stitched mainly for the purpose of reading the barcode and measuring liquid level. As long as the barcode reading and liquid level detection are right, some distortion and boundary effects are tolerated. Thus, the estimation of the perspective transform (both off-line and on-line) may be limited to a few of the eight parameters. However, in some cases, the parameters search must be expanded to all of the coefficients in order to read 2-D barcodes where the dimensions of the 2-D barcodes are larger than the overlap dimension between each tube rotation subimage, or where other important information, such as handwriting on the tube label, must be recorded.
