The present invention relates to a bar code reader having several cameras that are positioned within a housing for viewing objects moving past one or more windows to determine the content of bar codes on the objects.
A bar code is a coded pattern of graphical indicia having a series or pattern of bars and spaces of varying widths that encode information. Bar codes may be one dimensional (e.g., UPC bar code) or two dimensional (e.g., DataMatrix bar code). Systems that read, that is, image and decode bar codes employing imaging camera systems are typically referred to as imaging-based bar code readers or bar code scanners.
Imaging-based bar code readers may be portable or stationary. Stationary bar code readers are mounted in a fixed position and a target object, e.g., a product package that includes a target bar code, is moved or swiped past a transparent window so the object passes through a field of view of the stationary bar code reader. The bar code reader typically provides an audible and/or visual signal to indicate the target bar code has been successfully imaged and decoded.
A typical example where a stationary imaging-based bar code reader would be utilized includes a point of sale counter/cash register where customers pay for their purchases. The bar code reader is typically enclosed in a housing that is installed in a counter and normally includes a vertically oriented transparent window and/or a horizontally oriented transparent window, either of which may be used for reading the target bar code affixed to the target object, i.e., the product or product packaging. The sales person (or customer in the case of self-service check out) sequentially presents each target object's bar code either to the vertically oriented window or the horizontally oriented window, whichever is more convenient given the specific size and shape of the target object and the position of the bar code on the target object.
A stationary imaging-based bar code reader that has a plurality of imaging cameras can be referred to as a multi-camera imaging-based scanner or bar code reader. In a multi-camera imaging reader, each camera system typically is positioned so that it has a different field of view from every other camera system. While the fields of view may overlap to some degree, the effective or total field of view of the reader is increased by adding additional camera systems. Hence, the desirability of multi-camera readers as compared to single camera bar code readers which have a smaller effective field of view and require presentation of a target bar code to the reader in a very limited orientation to obtain a successful, decodable image.
If several imaging cameras are positioned to look out of the bar code reader's window or windows in different directions, the bar code reader can decode barcodes on various sides of packages that are swiped through the multiple, overlapping fields of view. Swiping promotes higher throughput and also assures that all the cameras have the opportunity to see their respective sides of a package, which may not happen if the package is stationary. Existing laser-based supermarket scanners are designed to handle swipe speeds up to 100 inches per second. With this high swipe speed, the barcode may only be within the field of view of any given camera for a single camera exposure frame, so it becomes important that the single frame be properly exposed, or the barcode might not be decodable.
One way to accommodate signal variations due to barcode distance and window composition is to use an auto-exposure system. Many camera image sensors include internal auto-exposure circuitry. This circuitry measures the level of light that is received on one frame and adjusts exposure duration on the next subsequent frame based on measurements taken on the first image or first frame. The result is that the exposure of the first frame may not be adequate to decode a barcode image but that the second captured frame should produce a decodable image.
Such adjustments are acceptable for some image readers, such as handheld bar code readers where the barcode remains stationary in front of the reader for several frame durations. Such an adjustment is not acceptable, however, where the user swipes the barcode rapidly through a field of view of the camera (or cameras) of a barcode reader. When swipe speed is high, the barcode can pass through the field of view before a second frame can be captured so it is important that the first frame captured be of the proper exposure for bar code interpretation.
U.S. Pat. No. 7,357,325 discusses an ‘Automatic Light Exposure Measurement and Illumination Control Subsystem’ that utilizes a rolling shutter with a single camera. It measures, in real-time, a power density [joules/em] of photonic energy (i.e. light) collected by the optics of the system at its image sensing array and generates Auto-Exposure Control Signals indicating the amount of exposure required for good image formation and detection. The system of this patent uses an optical band pass filter in the imaging system. An optical filter is needed with a rolling shutter since parts of the sensor will be exposing for long periods of time when the illumination is turned off, so these parts would provide blurred images if ambient light illuminated a moving barcode during that period.
Global shutters for use with imaging barcode readers and scan engines are known. The duration of the illumination matches or exceeds the integration or exposure time. Some global shutter scanners activate the illumination only during the integration time while others have the illumination activated continuously.
A real-time auto-exposure system for a bar code reader having multiple imagers is disclosed wherein a number of imagers or cameras use a global shutter sensor. Imaging sensors having global shutters can expose the sensor for a small portion of an overall image capture frame time. This capability allows the illumination systems of different cameras of the multiple camera system to flash, without interference. In the disclosed system, multiple global shutter sensors are programmed to operate at the longest exposure time that is anticipated to be necessary for the scanning application. For example, the programmed exposure time would be adequate to decode a barcode at the maximum required working range, assuming that the reader's illumination system is illuminated for that entire exposure time.
The disclosed system makes optimum use of a limited number of captured image frames. It allows the camera sensor to operate at a minimum number of frames per second since every image will be properly exposed. Auto exposure systems that can't assure proper exposure until a second frame, for example, will need to be operated at a higher frame rate to avoid missing a rapidly moving barcode. Sensors that operate at higher frame rates are more expensive, and the higher frame rate will place more of a load on the decoder, increasing its cost and power consumption.
The exposure time of the disclosed bar code reader is fast enough that it ends before the next camera flashes. For example, consider a bar code reader that has six cameras each operating at 60 frames/second. This allows each camera up to 2.7 ms exposure time before the next camera flashes, so each camera must use an exposure that is shorter than 2.7 ms. (6×23=16.4 ms×60 frames=984 ms) Exposure times of less than 500 microseconds can be achieved with a global shutter, so there is no chance that one camera will still be imaging when the next one flashes.
The disclosed system yields advantages not achievable by using a rolling shutter system and also not achievable when illuminating fields of view continuously during an entire integration time. In particular, a rolling shutter cannot be used in a scanner with multiple imagers each of which have their own illumination systems, and where the multiple cameras flash in succession in fractions of a frame time. If rolling shutters were used the image from one camera would be corrupted by flashes from another, since the rolling shutter would leave at least part of the frame exposing for an extended period of time, including when another of the cameras flashes its light. Since the fields of view overlap, each imager must end its exposure before the next one flashes so it is not influenced by the light projected by the other camera.
These and other objects, advantages, and features of the exemplary embodiment of the invention are described in detail in conjunction with the accompanying drawings.
In accordance with one use, either a sales person or a customer will swipe a product or target object 32 selected for purchase relative to the housing 20. A target bar code 30 imprinted or affixed to the target object 32 is swiped through a region near the windows H, V for reading, that is, imaging and decoding of the coded indicia of the target bar code.
Imaging Optics
An exemplary imaging system 12 has six cameras C1-C6 that capture a series of image frames in a programmable way under the control of a processor or processors 15. The series of image frames for each camera assembly C1-C6 is shown schematically as IF1, IF2, IF3, IF4, IF5, IF6 in
Digital signals 35 that make up the frames are coupled to a bus interface 42, where the signals are multiplexed and then communicated to a memory 44 in an organized fashion so that the processor knows which image representation belong to a given camera. Image processors 15 access the image frames IF1-IF6 from memory 44 and search for image frames that include an imaged target bar code 30′. If the imaged target bar code 30′ is present and decodable in one or more image frames, a decoder 16 attempts to decode the imaged target bar code 30′ using one or more of the image frames having the imaged target bar code 30′.
Each of the cameras has a two-dimensional (2D) CMOS image capture array with a typical size of the pixel array being on the order of 752×480 pixels. The illumination-receiving pixels of the sensor array define a sensor array surface secured to a printed, circuit board for stability. The sensor array surface is substantially perpendicular to an optical axis of the imaging lens assembly, that is, an axis that is perpendicular to the sensor array surface would be substantially parallel to the optical axis of the focusing lens. The pixels of the sensor array surface are disposed in an orthogonal arrangement of rows and columns of pixels.
A rolling shutter in a CMOS image sensor resets rows of pixels in sequence. As an example, the shutter starts at the top of the image and proceeds row by row to the bottom, resetting each row. When the reset process has moved some distance down the image, the readout process begins at the top. Rows of pixels are read out in sequence, starting at the top of the image and proceeding row by row to the bottom in exactly the same fashion and at the same speed as the reset process. The time delay between a row being reset and a row being read is the integration time. By varying the amount of time between when the reset sweeps past a row and when the readout of the row takes place, the integration time (hence the exposure) can be controlled for a sensor having a rolling shutter. Since the integration process moves through the image over some length of time, motion artifacts may occur and if a bar code is present on a moving object swiped in front of the reader, the bar code may not be decodeable.
A global shutter operates differently. When a global shutter is used, the entire image is reset before integration of the pixel contributions to the image. The pixels are allowed to accumulate charge during an integration time. At the end of the integration time, the accumulated charge in each pixel is simultaneously transferred to a light shielded storage area. Then the signals are read out from the light shielded area. Since all pixels are reset at the same time, integrated over the same interval, motion artifacts are avoided. A representative sensor having a global shutter is commercially available from Aptina (formerly Micron) under the part designation MT9V022.
An exemplary system uses one or more light emitting diodes closely spaced from an associated sensor array for each of the cameras C1-C6. The LEDs are selectively activated to emit light under control of the processors 15. This is shown more clearly in the schematic depiction of
Decoding Images
As is best seen in
The decoding circuitry 14 performs a process on selected image frames by getting an image from memory and determining if the image has a bar code. If so, a decoder 16 attempts to decode any decodable image within the image frames, e.g., the imaged target bar code 30′. If the decoding is successful, decoded data 56, representative of the data/information coded in the target bar code 30 is then output via a data output port 58 and/or displayed to a user of the reader 10 via a display 59. Upon achieving a good read of the target bar code 30, that is, the bar code 30 was successfully imaged and decoded, a speaker 34b and/or an indicator LED 34a is activated by the bar code reader circuitry 11 to indicate to the user that the target bar code 30 has been successfully read.
Multiple Camera Fields of View
Referring now to
The camera or imager C3 and its associated optics are symmetric with respect to a center line of the reader to imager C1. Camera C3 faces generally vertically upward toward an incline folding mirror M3A substantially directly overhead at a right side of the horizontal window H. The folding mirror M3A faces another inclined narrow folding mirror M3B located at a left side of the horizontal window H. The folding mirror M3B faces still another inclined wide folding minor M3C adjacent the mirror M3A. The folding mirror M3C faces out through the generally horizontal window H toward the left side of the dual window reader.
Imager or camera C2 and its associated optics are located between imagers C1 and C3 and their associated optics, Imager C2 faces generally vertically upward toward an inclined folding mirror M2A substantially directly overhead generally centrally of the horizontal window H at one end thereof. The folding mirror M2A faces another inclined folding mirror M2B located at the opposite end of the horizontal window H. The folding mirror M2B faces out through the window H in an upward direction toward the vertical window V in the housing 20.
As illustrated in
In
In
Features and functions of the fold mirrors shown in
Each of the multiple cameras has its own dedicated light source for illuminating that camera's field of view. The light source is only energized when its associated camera is capturing an image. Turning to
The illumination intensity provided by the diodes 120, 122 is bright enough that the maximum required exposure time is shortened and terminated before the next camera begins to capture an image. Therefore, the first camera C1 will no longer be exposing when the illumination system for the next camera C2 is activated, and the exposure of the first camera will not be affected by the illumination flash of the second camera system. The imagers are exposed in a sequence 140 (
This time slicing allows the cameras C1-C6 to operate at an exposure time that is sufficient, but not much greater than is necessary to produce a decodable image at maximum desired range. At close range, however, this exposure time will be longer than necessary, and may be too long to allow the desired swipe speed due to image blur. It is therefore necessary to reduce the exposure time when barcodes are positioned closer to the scanner, and this reduction must be accomplished without wasting any frames.
A photo-detector sensor 150 (
When the diodes 120, 122 are energized, the output from the auto-exposure phototdetector or sensor 150 is integrated until the integrator exceeds a threshold value, at which time the diodes 120, 122 are deactivated. One suitable photodetector is commercially available as part no. BPW34 by Osram. A threshold value is set to correspond to the correct exposure level for the sensor 124. The image sensor 124 continues its exposure until its preset exposure time is reached, but since the illumination from the LEDs 120, 122 has been turned off, no significant amount of additional light will be collected.
Use of a global shutter assures that the time during which the sensor 124 is exposing is minimized, so the opportunity for interference from ambient light is far reduced. With a rolling shutter, at least part of the frame may be exposing for as much as 30 ms (the rolling shutter sensor runs at a maximum of 30 frames/second), but the required exposure time, and therefore the duration of the illumination flash may be as little as, for example, 300 us. The image that is captured during the flash of illumination therefore can be corrupted during the remainder of the 30 ms exposure time.
When a sensor 124 with a global shutter is used, the time during which the illumination flash has ended but the sensor is still exposing is only perhaps 100 us, assuming the maximum exposure time has been set to 400 us, so ambient light will have little influence on the ultimate brightness of the image. No optical filter is needed since the influence of ambient light has already been minimized.
The maximum exposure time of the reader 10 is changed based on the light transmission efficiency of the scan window H, V. The light reflected off from the barcodes and received by the imagers can vary due to variations in distance from the scanner or variations in the kind of window that is installed on the scanner. For example scanner windows might be tempered glass which transmits around 90% of incident light. Windows made from glass with a scratch resistant diamond-like coating (DLC) transmits light at a 55% efficiency. The window or windows can include a thin coating of sapphire laminated to a glass substrate which transmit light with 72% efficiency at a wavelength of 630 nm, the wavelength of LED illumination systems used with exemplary cameras. These transmission efficiencies are typical for light passing through a window at 45 degrees, which is common for existing imagers.
If a DLC window is being used, maximum exposure time can be set, for example, to 400 usecs, but when a sapphire window is used maximum time can be set to 250 usecs, since the superior light transmission of the sapphire window will never require a longer exposure time. Reducing the maximum exposure time when a higher transmission window is used minimizes the impact of ambient light, which is transmitted more easily through the higher transmission window, and also avoids increasing the duration of an illumination flash more than necessary when no object is within the field of view, in which case the auto-exposure system will turn the flash duration up to maximum, since no reflected light is detected. Regardless of the maximum exposure time, both the actual shutter exposure time of the sensor and the maximum illumination flash duration will be set to be around the same. The auto-exposure control implemented by the processors can reduce illumination flash duration, but would generally not be allowed to produce a flash whose duration exceeds the maximum exposure duration, since any such prolonged flash is wasted, annoying to the user and needlessly increases power consumption of the scanner.
In order to utilize different maximum exposure times for different kinds of windows, the scanner must be able to identify which window has been installed. This can be accomplished several ways. A barcode 160 (
In some multi-camera bar code readers auto-exposure cameras are needed on only a subset of the imagers. As discussed above in the disclosed reader 10, three of the cameras obtain images through the vertical window V and three other cameras obtain images through the horizontal window H. The vertical window V is not subject to the same abuse as the horizontal windows, which must survive even though metal objects such as cans of food and tools will be dragged across it. Therefore scratch resistant window materials such as DLC and Sapphire are not generally used for the vertical window in such bar code readers. Vertical windows can be made, for example, of tempered glass, which has better optical transmission characteristics than either Sapphire or DLC. With this superior optical transmission, it is possible to project enough light to use a very short exposure time even when scanning at maximum range. There is therefore no need to increase exposure time to achieve full range, as is necessary when scanning through windows with lower transmission. Imagers that look through high transmission windows can therefore operate with fixed exposure times, in which the flash duration and sensor shutter exposure time are matched at a preset value. This eliminates the need to implement auto exposure with some of the imagers, reducing system cost.
A bar code reader built in accordance with this invention can therefore use auto exposure on only a subset of the cameras. Alternatively, all imagers can use an auto-exposure system as described above, including the imagers that look through a high transmission window. Providing these imagers with an auto-exposure system will cause them to operate with shorter exposure durations when barcodes are scanned close to the window, as is commonly the case. Shorter exposures increase the scanners ability to capture undistorted images of moving objects, so the addition of the auto-exposure system will increase the allowable swipe speed when barcodes are being scanned at closer then maximum range.
What have been described above are examples of the present invention. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the present invention, but one of ordinary skill in the art will recognize that many further combinations and permutations of the present invention are possible. Accordingly, the present invention is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims.
This application is a continuation of U.S. patent application Ser. No. 12/417,287, field Apr. 2, 2009, titled “AUTO-EXPOSURE FOR MULTI-IMAGER BARCODE READER.”
Number | Name | Date | Kind |
---|---|---|---|
4613895 | Burkey et al. | Sep 1986 | A |
4794239 | Allais | Dec 1988 | A |
4818856 | Matsushima et al. | Apr 1989 | A |
5059779 | Krichever et al. | Oct 1991 | A |
5124539 | Krichever et al. | Jun 1992 | A |
5200599 | Krichever et al. | Apr 1993 | A |
5304786 | Pavlidis et al. | Apr 1994 | A |
5331142 | Takenaka | Jul 1994 | A |
5510607 | Ishikawa | Apr 1996 | A |
5559562 | Ferster | Sep 1996 | A |
5703349 | Meyerson et al. | Dec 1997 | A |
5717195 | Feng et al. | Feb 1998 | A |
5869827 | Rando | Feb 1999 | A |
6065678 | Li et al. | May 2000 | A |
6141062 | Hall et al. | Oct 2000 | A |
6189795 | Ohkawa et al. | Feb 2001 | B1 |
6296187 | Shearer | Oct 2001 | B1 |
6924807 | Ebihara et al. | Aug 2005 | B2 |
7076097 | Kondo et al. | Jul 2006 | B2 |
7116353 | Hobson et al. | Oct 2006 | B2 |
7191947 | Kahn et al. | Mar 2007 | B2 |
7280124 | Laufer et al. | Oct 2007 | B2 |
7296744 | He et al. | Nov 2007 | B2 |
7344080 | Vinogradov et al. | Mar 2008 | B2 |
7357325 | Zhu et al. | Apr 2008 | B2 |
7430682 | Carlson et al. | Sep 2008 | B2 |
7527207 | Acosta et al. | May 2009 | B2 |
7757955 | Barkan | Jul 2010 | B2 |
7823787 | He et al. | Nov 2010 | B2 |
8146821 | Barkan et al. | Apr 2012 | B2 |
20010042789 | Krichever et al. | Nov 2001 | A1 |
20020038820 | Check et al. | Apr 2002 | A1 |
20020056750 | Kato et al. | May 2002 | A1 |
20020125317 | Hussey et al. | Sep 2002 | A1 |
20020179713 | Pettinelli et al. | Dec 2002 | A1 |
20030029915 | Barkan et al. | Feb 2003 | A1 |
20030090805 | Ohkawa et al. | May 2003 | A1 |
20040146211 | Knapp et al. | Jul 2004 | A1 |
20050259746 | Shinde et al. | Nov 2005 | A1 |
20060022051 | Patel et al. | Feb 2006 | A1 |
20060118627 | Joseph et al. | Jun 2006 | A1 |
20060118628 | He et al. | Jun 2006 | A1 |
20070079029 | Carlson et al. | Apr 2007 | A1 |
20070176003 | Brock | Aug 2007 | A1 |
20080105745 | Lei | May 2008 | A1 |
20080179402 | Barkan et al. | Jul 2008 | A1 |
20090020611 | Sackett et al. | Jan 2009 | A1 |
20090020612 | Drzymala et al. | Jan 2009 | A1 |
20090026271 | Drzymala et al. | Jan 2009 | A1 |
20090078775 | Giebel et al. | Mar 2009 | A1 |
20090084854 | Carlson et al. | Apr 2009 | A1 |
20090101719 | Knowles et al. | Apr 2009 | A1 |
20090159683 | Roquemore et al. | Jun 2009 | A1 |
20090206161 | Olmstead | Aug 2009 | A1 |
20090272810 | Barkan | Nov 2009 | A1 |
20100019042 | Barkan | Jan 2010 | A1 |
20100019043 | Sackett et al. | Jan 2010 | A1 |
20100102129 | Drzymala et al. | Apr 2010 | A1 |
20100116887 | Barkan et al. | May 2010 | A1 |
20100147953 | Barkan | Jun 2010 | A1 |
20100155484 | Gregerson | Jun 2010 | A1 |
20100187313 | He et al. | Jul 2010 | A1 |
20100200656 | Marshall et al. | Aug 2010 | A1 |
20100252633 | Barkan et al. | Oct 2010 | A1 |
20100252635 | Drzymala et al. | Oct 2010 | A1 |
20110024505 | Wang et al. | Feb 2011 | A1 |
20110127333 | Veksland et al. | Jun 2011 | A1 |
Number | Date | Country |
---|---|---|
2006062818 | Jun 2006 | WO |
Entry |
---|
International Preliminary Report on Patentability for International Application No. PCT/US2010/029001 mailed on Oct. 13, 2011. |
International Search Report and Written Opinion for International Application No. PCT/US2010/029001 mailed on Aug. 20, 2010. |
Non Final Office Action mailed Jun. 24, 2011 in counterpart U.S. Appl. No. 12/417,287, filed Apr. 2, 2009. |
Notice of Allowance mailed Dec. 21, 2011 in counterpart U.S. Appl. No. 12/417,287, filed Apr. 2, 2009. |
Number | Date | Country | |
---|---|---|---|
20120160917 A1 | Jun 2012 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 12417287 | Apr 2009 | US |
Child | 13411815 | US |