Patent Application
20220327325
The invention is directed to optical image sensors, and more particularly, to optical image sensors which use captured images to perform object detection.
In standard object detection, proximity sensors, photoelectric sensors or ultrasonic sensors are used to determine the presence of an object within a certain range.
Although these sensors work well for object detection, the circuitry can be complicated and requires large power resources.
This in mind, it is an objective of the present invention to provide a system and method for object detection using captured raw images taken by an image sensor.
An object detection system and method comprises: an optical image sensor arranged to perform the following steps: capturing a calibration image during a calibration stage, dividing the calibration image into a plurality of quadrants, and calculating a parameter for each of the quadrants; capturing a plurality of raw images during a detection stage, dividing each image of the raw images into a plurality of quadrants, and calculating a parameter for each of the quadrants; comparing the respective parameters of each quadrant of a raw image with the respective parameters of each quadrant of the calibration image to generate a ratio value for each quadrant; and comparing the ratio value of each quadrant with a predetermined threshold. When each ratio value of specific quadrants of the quadrants is greater than the predetermined threshold, object detection is confirmed. The system further comprises a plurality of infrared LEDs for providing infrared illumination to the system and an IR filter positioned in front of the optical image sensor for filtering the infrared illumination. The infrared LEDs provide infrared illumination to the system when an ambient light of the system is below a particular threshold.
The parameter may be pixel intensity, a sum of pixel intensities within each quadrant, or an average of pixel intensities within each quadrant. The specific quadrants may be the central quadrants. The predetermined threshold can be adjusted according to a parameter of the calibration image, wherein the parameter is noise.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
In contrast to the prior art which uses sensors to perform object detection, the method and system of the present invention uses raw images captured by an image sensor to perform object detection. In this way, circuitry can be kept simple while still achieving the objectives of object detection.
Refer to
The following description uses a coffee machine detecting the presence of a coffee cup as an example, but this is merely for best illustration of the inventive features, and the invention is not limited to this particular embodiment. The optical image sensor 50 illustrated above is used to take images for detecting whether a coffee cup is present or not. In this embodiment, the optical image sensor 50 will be positioned directly above where a coffee cup should be placed, i.e. on a flat tray below a spout of the coffee machine for distributing coffee (or other liquids).
As will be understood, it is desirable for a coffee machine to be able to detect the presence of a coffee cup, wherein if no coffee cup is present or if the coffee cup is not centred underneath the coffee machine then the coffee machine will not pour coffee. In order to perform the method of the present invention, a calibration image corresponding to a situation where no coffee cup is present will first be captured.
Refer to
The system then enters a detection stage, wherein raw images are captured by the optical image sensor 50 and used to determine whether an object is present. Refer to
In order to accurately determine the presence of an object, for each raw image captured during the detection stage, the sum of values for each quadrant will be compared to the calibration raw image.
This is illustrated in
Ratio of quadrant=absolute(background quadrant−new quadrant)/background quadrant×100%
As shown in the quadrant ratio chart 490, the four quadrants at the centre of the image have a ratio close to or more than 100%. Using this data, it can be determined that there is an object present in the centre of the raw image 330. In an exemplary embodiment, a threshold value for determining the presence of an object is set to 20%. Please note that this threshold may be set to any value, and the particular value may depend on how much ‘noise’ is present in the calibration image 230. For a ‘noisy’ image, it is preferable to set the threshold to a higher value. The calibration stage may also be performed for a plurality of images to determine the amount of noise with greater accuracy.
The above method depends upon a comparison between the calibration image and raw images captured during a detection stage. The value of the parameters calculated for each quadrant of a raw image will not only depend on the presence or lack of an object, but also on the ambient lighting. As will be known by those skilled in the art, an image obtained in bright conditions will be very different from an image obtained in dark conditions. Furthermore, reflectivity of the surface being photographed will also influence the raw data.
Rather than adjusting calibration and thresholds depending on the ambient environment, it is more convenient to control the environment in which the raw images are being captured. The image sensor system 100 therefore further includes the IR filter 30 and the IR LEDs 70 illustrated in
Step 600: Begin calibration. Turn on the LEDs and use an image sensor to capture an image of a flat surface;
Step 602: Divide the captured image into quadrants;
Step 604: Calculate a value of each quadrant according to a specific parameter. Enter normal flow;
Step 606: Continuously use the image sensor to capture images of the flat surface, and perform steps 602 and 604 for the captured images;
Step 608: Calculate a ratio of the calibration image parameters compared to the captured image parameters;
Step 610: Are any specific quadrants reserved for image detection? If yes, go to Step 612; if no, go to Step 614;
Step 612: Do any of the specific quadrants contain a value greater than a threshold? If yes, go to Step 614; if no, return to Step 606;
Step 614: Do any of the quadrants contain a value greater than a predetermined threshold? If yes, go to Step 616; if no, return to Step 606;
Step 616: Object detection confirmed.
The above method and system are described with reference to a coffee machine for detecting a coffee cup, but the method and system can be applied to any situation for detecting an object using an optical image sensor. By using an optical image sensor to capture images, dividing the captured images into quadrants, calculating a value of each quadrant corresponding to a particular parameter, and comparing the values of a captured image with those of a calibration image, the above system and method can accurately determine the presence of an object using an optical image sensor alone. The system and method can also set certain quadrants only to give a positive result for object detection, and can further set different thresholds and/or incorporate IR lighting to compensate for ambient lighting and changes in the local environment.
Thus, the above method and system for determining the presence of an object can yield good results without requiring large power resources or costs.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.
