This application claims the benefit of Taiwan application Serial No. 107136247, filed Oct. 15, 2018, the subject matter of which is incorporated herein by reference.
The disclosure relates in general to a dual-view image device for monitoring a heat source and an image processing method thereof.
As safety consciousness gradually arises, the application of security surveillance also receives more and more attention. To avoid having dead ends within the surveillance area, normally multiple monitors or cameras are installed to monitor different regions simultaneously.
The user can install multiple monitors in multiple interested regions of the surveillance area, such that the image of each interested region can be sent to a central console where the control personnel can perform a comprehensive surveillance to assure that no dead end appears. However, the current monitor only has mono lens and mono view, and is unable to perform multi-image detection and all-day monitoring for the same surveillance area. Particularly, when an object invades the surveillance area during the nighttime, the watcher is unable to track the invader in the image or lacking a partial enlargement of the image, can only rely on the monitors installed at different places to recognize the invader in the image. Therefore, the current monitor cannot efficiently provide clear images for the user's viewing and reference.
The present disclosure relates to a dual-view image device for monitoring a heat source. The dual-view image device can track the heat source within a surveillance area and provides a dual-view thermal image for the user's viewing and reference.
According to one embodiment of the present disclosure, a dual-view image device for monitoring a heat source is provided. The dual-view image device includes a first thermal image sensor, a second thermal image sensor, a driver and a processor. The first thermal image sensor has a first view angle and is configured to capture a first thermal image having a heat source within the first view angle. The second thermal image sensor has a second view angle and is configured to capture a second thermal image having the heat source within the second view angle. The second view angle is smaller than the first view angle. The pixels occupied by the heat source in the first thermal image are fewer than that occupied by the heat source in the second thermal image. The driver is configured to drive the first thermal image sensor and the second thermal image sensor to track and position the heat source within the first and the second view angles. The processor is configured to combine the first thermal image and the second thermal image into a dual-view image and output the dual-view image having the heat source.
According to another embodiment of the present disclosure, a dual-view image processing method for monitoring a heat source is provided. The dual-view image processing method includes the following steps. Firstly, a heat source is detected by a first thermal image sensor having a first view angle, wherein the first thermal image sensor captures a first thermal image having the heat source within the first view angle. Then, the heat source is detected by a second thermal image sensor having a second view angle, wherein the second view angle is smaller than the first view angle, the second thermal image sensor captures a second thermal image having the heat source within a second view angle, and the pixels occupied by the heat source in the first thermal image are fewer than that occupied by the heat source in the second thermal image. Then, the first thermal image sensor and the second thermal image sensor are driven to track and position the heat source within the first and the second view angle. Then, the first thermal image and the second thermal image are combined into a dual-view image and the dual-view image having the heat source is outputted.
The above and other aspects of the disclosure will become understood with regard to the following detailed description of the preferred but non-limiting embodiment(s). The following description is made with reference to the accompanying drawings.
Detailed descriptions of the disclosure are disclosed below with a number of embodiments. However, the disclosed embodiments are for explanatory and exemplary purposes only, not for limiting the scope of protection of the disclosure. Similar/identical designations are used to indicate similar/identical elements. Directional terms such as above, under, left, right, front or back are used in the following embodiments to indicate the directions of the accompanying drawings, not for limiting the present disclosure.
According to an embodiment of the present disclosure, a dual-view image device for monitoring a heat source is provided. The dual-view image device includes two thermal image sensors having different view angles for monitoring a target heat source (such as a human or an animal) within a surveillance area. When the target heat source is located within the visible range of large view angle, the dual-view thermal image sensors can be simultaneously moved or the image sensor having a small view angle can be moved to track the heat source, such that the target heat source can be simultaneously located within both the visible ranges of large view angle and small view angle. Then, the dual-view images having the heat source are combined by the processor and outputted to a display device for the control personnel to determine the location of the heat source or for the monitoring system to send an abnormality warning to notify the control personnel.
Refer to
The first thermal image sensor 110, which can be made by a wide-angle camera with image tracking function, is disposed on the driver 130. The first thermal image sensor 110 can scan a surveillance area at a specific location to detect whether a heat source P appears within the surveillance area. The wide-angle camera, having a wide angle such as 100 degrees or above, can perform a full-range scan on the surveillance area of large view angle to avoid having any dead ends within the surveillance area.
The second thermal image sensor 120, which can be made by a narrow-angle camera with image tracking function, is disposed on the driver 130. The first thermal image sensor 110 and the second thermal image sensor 120 can be co-axially disposed on the driver 130 and simultaneously rotated by the same angle to track a heat source P. In another embodiment, the first thermal image sensor 110 and the second thermal image sensor 120 can be hetero-axially disposed on the driver 130 and rotated by respectively angles to track the heat source P. Since the view angle of the second thermal image sensor 120 is smaller than that of the first thermal image sensor 110, the second thermal image sensor 120 can detect a partial small-angle region, particularly the partial region where the heat source P appears, such that the scope of surveillance can be reduced. Meanwhile, the second thermal image sensor 120 can track the heat source P and provide a partial image.
In an embodiment, when it is detected that the heat source P appears within a surveillance area, the first thermal image sensor 110 calculates the feature point coordinates T, such as the center point coordinates, of the heat source P for the second thermal image sensor 120 to track the heat source P. The second thermal image sensor 120 can locate the local region of the heat source P according to the feature point coordinates T of the heat source P and perform a small-angle scan to determine whether the heat source P still appears in the region of small view angle.
The driver 130, such as a motor and a gear set, is configured to simultaneously or individually drive the first thermal image sensor 110 and the second thermal image sensor 120. The driver 130 can also be an actuator for simultaneously or individually actuating the first thermal image sensor 110 and the second thermal image sensor 120. In an embodiment, the surveillance system can determine the feature point coordinates T of the heat source P according to the feedback of an image signal and generate a drive command to the driver 130 to simultaneously or individually rotate the first thermal image sensor 110 and the second thermal image sensor 120 to the angle and orientation of the local region of the heat source P. Therefore, the first thermal image sensor 110 and/or the second thermal image sensor 120 can be moved to a pre-detected position to track the heat source P according to the drive command, such that the heat source P is located within the view angle (or visible range) of the first thermal image sensor 110 and/or the second thermal image sensor 120.
Refer to
Refer to
The processor 140 is configured to combine the first thermal image M1 and the second thermal image M2 into a dual-view image Mx and output the dual-view image Mx having the heat source P to the display device 150 for the user's viewing and reference. In an embodiment, the processor 140 not only combines the first thermal image M1 and the second thermal image M2 into a dual-view image Mx but further synchronizes the coordinate information of the first thermal image M1 and the second thermal image M2. The processor 140, which can be exemplified by such as a digital image mixer or a multiplex processor, can combine the thermal images of large view angle and of small view angle to satisfy the requirements of security surveillance with the dual-view thermal image.
Refer to
Using the above image processing technique, the user can view both the heat source P within the first view angle θ1 and the heat source P within the second view angle θ2, wherein the heat source image P within the second view angle θ2 is relatively larger than the heat source image P within the first view angle θ1, such that the user can more conveniently recognize detailed features, such as eyes, nose and mouth, of the heat source P.
Moreover, before the first thermal image M1 and the second thermal image M2 are combined into the dual-view image Mx, the quality of the first thermal image M1 and/or that of the second thermal image M2 can further be increased using sub-pixel complement to increase the image resolution. The image processing method for sub-pixel complement includes the following steps. Firstly, a thermal image with low resolution is obtained, and corresponding coordinates of each pixel are calculated according to the stitched sequence of the thermal image with low resolution. Then, the corresponding coordinates of the pixels in the thermal image with high resolution are supplemented according to the corresponding coordinates of each pixel. The unknown pixels in the thermal image with high resolution are supplemented using average values of the known pixels. Detailed steps of the dual-view image processing method are disclosed below.
Refer to
As indicated in
Refer to
As indicated in
According to the dual-view image device for monitoring a heat source and an image processing method thereof disclosed in above embodiments of the present disclosure, the heat source can be monitored and tracked in the full range, a partial image of the local region of the heat source can be provided, and the dual-view thermal image sensors can be simultaneously moved to track the heat source or the small view angle image sensor can be individually moved to track the heat source, such that target heat source is located within both the visible ranges of the large view angle and the small view angle. Thus, the dual-view image device of the present embodiment can detect the same surveillance area from multi-angles for the user's viewing and reference to satisfy the requirements of security surveillance with multi-view thermal image.
While the disclosure has been described by way of example and in terms of the preferred embodiment(s), it is to be understood that the disclosure is not limited thereto. On the contrary, it is intended to cover various modifications and similar arrangements and procedures, and the scope of the appended claims therefore should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements and procedures.
Number | Date | Country | Kind |
---|---|---|---|
107136247 | Oct 2018 | TW | national |
Number | Name | Date | Kind |
---|---|---|---|
5299971 | Hart | Apr 1994 | A |
6414314 | Ikeda | Jul 2002 | B1 |
7346188 | Aichi | Mar 2008 | B2 |
9330312 | Brown et al. | May 2016 | B2 |
9483952 | Chew | Nov 2016 | B2 |
9871999 | Kardashov | Jan 2018 | B2 |
9875556 | McNally | Jan 2018 | B2 |
9887775 | Puscasu et al. | Feb 2018 | B2 |
9888073 | Ohana et al. | Feb 2018 | B2 |
10863098 | Priyantha | Dec 2020 | B2 |
20060175549 | Miller | Aug 2006 | A1 |
20070235648 | Teich | Oct 2007 | A1 |
20080260225 | Szu | Oct 2008 | A1 |
20140139643 | Hogasten | May 2014 | A1 |
20150242683 | Tang | Aug 2015 | A1 |
20180096468 | Nguyen | Apr 2018 | A1 |
20180372548 | Gosian | Dec 2018 | A1 |
Number | Date | Country |
---|---|---|
1279751 | Oct 2006 | CN |
100508598 | Jul 2009 | CN |
103168467 | Jun 2013 | CN |
103595916 | Feb 2014 | CN |
103986917 | Apr 2017 | CN |
572531 | Jan 2004 | TW |
201205115 | Feb 2012 | TW |
I607276 | Dec 2017 | TW |
WO 2005036371 | Apr 2005 | WO |
Entry |
---|
Taiwanese Office Action and Search Report dated Jul. 2, 2019 for Application No. 107136247. |
Kumar et al., “Improving Person Tracking Using an Inexpensive Thermal Infrared Sensor”, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2014, pp. 217-224. |
Ma et al., “Pedestrian Detection and Tracking from Low-Resolution Unmanned Aerial Vehicle Thermal Imagery”, Sensors, 2016, No. 16, vol. 446, total 26 pages. |
Mählisch et al., “A Multiple Detector Approach to Low-resolution FIR Pedestrian Recognition”, 2005, total 6 pages. |
Portmann et al., “People Detection and Tracking from Aerial Thermal Views”, 2014 IEEE International Conference on Robotics & Automation (ICRA), Hong Kong Convention and Exhibition Center, May 31-Jun. 7, 2014. Hong Kong, China, pp. 1794-1800. |
Teutsch et al., “Low Resolution Person Detection with a Moving Thermal Infrared Camera by Hot Spot Classification”, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2014, pp. 209-216. |
CN Office Action 2018112942064 dated Sep. 27, 2020; pp. 1-8. |
Number | Date | Country | |
---|---|---|---|
20200120290 A1 | Apr 2020 | US |