Embodiments of the present disclosure relates to the field of artificial intelligence, especially to computer vision, and in particular to a method and an apparatus for measuring temperature, and a computer-readable storage medium.
High-density flow of people generally appears at an entrance of public transport or an entrance of a scenic spot or a venue with the development of economy and transportation construction in China. Due to the possible and long-term epidemic situation of infectious diseases, temperature measurement and recognition of identity information for the high-density flow of people are important links in epidemic prevention and control. However, the existing non-contact temperature measurement method is usually that security and epidemic prevention personnel detect a temperature of pedestrians one by one through thermometers, and recognize an identity of the pedestrians one by one through scanning ID cards and other ways. The method is obviously inefficient. Moreover, due to the possible congestion of the flow of people, gathering of the high-density flow of people is generally inevitable, thus causing more people to be infected. How to effectively and accurately measure the temperature is an urgent problem for the epidemic prevention and control.
According to embodiments of the present disclosure, a solution for measuring temperature is provided.
In a first aspect of the present disclosure, a method for measuring temperature is provided. The method includes detecting a target position of an object in an input image. The method further includes determining key points of the target position and weight information of each key point based on a detection result of the target position, in which the weight information is configured to indicate a probability of each key point being covered. The method may further include acquiring temperature information of each key point. In addition, the method may further include determining a temperature of the target position at least based on the temperature information and the weight information of each key point.
In a second aspect of the present disclosure, an apparatus for measuring temperature is provided. The apparatus includes one or more processors, and a memory configured to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement: detecting a target position of an object in an input image; determining key points of the target position and weight information of each key point based on a detection result of the target position, in which the weight information is configured to indicate a probability of each key point being covered; acquiring temperature information of each key point; and determining a temperature of the target position at least based on the temperature information and the weight information of each key point.
In a third aspect of the present disclosure, a computer-readable storage medium having a computer program stored thereon is provided. The program is caused to implement a method for measuring temperature when executed by a processor. The method includes: detecting a target position of an object in an input image; determining key points of the target position and weight information of each key point based on a detection result of the target position, in which the weight information is configured to indicate a probability of each key point being covered; acquiring temperature information of each key point; and determining a temperature of the target position at least based on the temperature information and the weight information of each key point.
It should be understood that, the contents described in the Summary are not intended to identify key or important features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the disclosure will become apparent from the following description.
The above and other features, advantages and aspects of respective embodiments of the disclosure will become more apparent with reference to accompanying drawings and following detailed illustrations. In the accompanying drawings, the same or similar numeral references represent the same or similar elements, in which:
Embodiments of the disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the disclosure are illustrated in the accompanying drawings, it should be understood that the disclosure may be implemented in various manners without being limited by the embodiments elaborated herein. On the contrary, embodiments are provided to make the disclosure more thorough and complete. It should be understood that, the accompanying drawings and embodiments of the disclosure are merely used for exemplary purposes, and are not used to limit the protection scope of the disclosure.
In the description of embodiments of the disclosure, the term “includes” and its equivalents should be understood as an open “include” (a non-exclusive “include”), that is, “include but not limited to”. The term “based on” should be understood as “based at least in part (at least partially based on)”. The term “an embodiment” or “the embodiment” should be understood as “at least one embodiment”. The terms “first”, “second”, and the like may refer to different or the same objects. Other explicit and implicit definitions may also be included below.
In order to deal with epidemic prevention and control, it may generally measure temperatures of a plurality of pedestrians simultaneously using a non-contact and automated temperature measurement method based on a thermal imaging technology. For example, an infrared thermal imaging image of a face of a pedestrian may be acquired through an infrared thermal imaging device. When the infrared thermal imaging image shows that the face of the pedestrian is in a high temperature state (for example, a forehead is displayed in red), a detection device may determine that the pedestrian is in a fever state and send an alarm message.
However, temperature measurement accuracy of this type of temperature measurement technology is easily influenced by surrounding environments. For example, the surrounding of the face of the pedestrian may be covered by a high-temperature object (such as a mobile phone, or a hot drink, etc.). For another example, the surrounding of the face of the pedestrian may be covered by a low-temperature object (such as cold food, a cold drink, etc.). For yet another example, a face of one fever free pedestrian may be covered by a fever pedestrian. These cover conditions generally lead to false temperature alarm, thus affecting performance of the detection device. In addition, when the face is covered, traditional face recognition mechanisms may output an inaccurate recognition result, also.
As mentioned above, there is an urgent need for a method of measuring temperature to acquire temperature information of the pedestrians quickly, effectively and at low-costly, thus reducing time and labor costs of temperature measurement.
According to an embodiment of the present disclosure, a solution for measuring temperature is proposed. In this solution, key points of a target position of an object may be determined based on an input image acquired by a camera, and weight information of each key point may be further determined. Herein, the weight information is configured to indicate a probability of each key point being covered. On the other hand, temperature information at each key point may be acquired by a temperature sensing device, such as an infrared thermal imaging device, and a temperature of the target position may be determined based on the temperature information and the weight information of each key point. In this way, even if there is a situation such as the target position of a face of a person is covered by a high-temperature object (such as a mobile phone, a hot drink, etc.), the weight information of each key point of the covered position may be determined to be very small or may be ignored. Therefore, a temperature of a pedestrian may be determined more based on temperatures of positions that are not covered or are not affected by other abnormal temperatures.
Description will be made in detail below to embodiments of the disclosure with reference to accompanying drawings.
The input image 110 may be a real-time monitoring image acquired by an image acquisition device connected to the computing device 120. As an example, the image acquisition device may be arranged in a public place with a high flow of people to acquire image information of each person in the crowd passing through the place. It may be understood that acquiring the object from the image information may not be limited to persons, but also include animals that require to measure their temperature in batches (for example, animals in a zoo or a feeding station). In addition, the input image 110 may also be multiple frames of images of a monitored object, namely, a video. The computing device 120 may receive the input image 110, and determine a detection region for the target position, such as a face, of the monitored object through the CNN 140 in the computing device 110, and further determine the key points and the weight information of each key point.
On the other hand, the computing device 120 also receives a temperature sensing image 130. The temperature sensing image 130 may be acquired by a temperature sensing device, such as an infrared thermal imaging device. The temperature sensing device such as the infrared thermal imaging device may register with the image acquisition device described above, to achieve pixel-level alignment of imaging of the two devices. Thus, the computing device 120 may determine the temperature information of each key point, and determine a temperature 150 of the monitored object based on the temperature information of each key point with more reference significance by means such as weighted average.
In
As mentioned above, a process of determining a detection region for a target position, such as a face, of a monitored object, and a process of determining key points and weight information of each key point each may be divided into two phases including a model training stage and a model application stage. As an example, for the process of determining the detection region for the target position, in the model training phase, the model training system 270 may train a CNN 240 for determining the detection region through a training dataset 260. In the model application phase, the model application system 280 may receive a trained CNN 240, such that the CNN 240 may determine the detection region based on the input image 210. It may be understood that the training dataset 260 may be a mass of labeled monitoring images.
As another example, for the process of determining the key points and the weight information of each key point, in the model training phase, the model training system 270 may train a CNN 240 for determining the key points and the weight information of each key point through the training dataset 260. In the model application phase, the model application system 280 may receive a trained CNN 240, such that the CNN 240 may determine the key points and the weight information of each key point based on the determined detection region. It may be understood that the CNN 240 may also be trained to determine the key points and the weight information of each key point directly based on the input image 110.
In other embodiments, the CNN 240 may be constructed as a learning network for determining the key points and the weight information of each key point. Such a learning network may also be called a learning model, or simply a network or a model. In some embodiments, the learning network for determining the key points and the weight information of each key point may include a plurality of networks. Each network may be a multi-layer neural network, which may be composed of a large number of neurons. Through the training process, corresponding parameters of the neurons in each network may be determined. The parameters of the neurons in these networks are collectively referred to as parameters of the CNN 240.
A training process of the CNN 240 may be performed in an iterative manner. Specifically, the model training system 270 may acquire a reference image from the training dataset 260, and perform one iteration of the training process using the reference image to update the corresponding parameters of the CNN 240. The model training system 270 may repeat the above process based on a plurality of reference images in the training dataset 260 until at least some of the parameters of the CNN 240 converge, thus acquiring final parameters of the model.
The technical solution described above is only for illustrative purposes only not to limit the present disclosure. It may be understood that each network may also be arranged in other ways and connection relationships. In order to explain the principle of the above solution more clearly, the process of measuring temperature will be described in detail below with reference to
At 302, a computing device 120 may detect a target position of an object in an input image 110. As an example, the computing device 120 may determine a detection region for the target position in the input image 110 through a CNN 140 (such as a detection region generation model). In some embodiments, the CNN 140 may perform face region detection on the input image 110. For example, basic face features may be extracted from the input image 110 through a six-layer convolution network, and each layer of the convolution network may realize once sampling for the image. The last three layers of convolution network may be respectively preset a fixed number of different sizes of face anchor regions to perform regression of a detection region for a face, and finally the detection region for the face is determined. It may be understood that the above example is illustrative, and may use other layers of convolutional networks, and are not limited to determining the detection region for the face. In this way, the detection area of the target position in the input image 110 may be quickly recognized based on the detection area generation model, which may prepare for the subsequent temperature measurement and even face recognition.
At 304, the computing device 120 may determine key points of the target position and weight information of each key point based on a detection result of the target position. The weight information is configured to indicate a probability of each key point being covered. As an example, the computing device 120 may apply the detection result of the target position to the CNN 140 (such as a key point determination model) to determine the key points and the weight information of each key point. The CNN 140 is obtained by training based on a reference target position in a reference image, and reference key points of the reference target position and reference weight information. In some embodiments, the CNN 140 may determine the key points of the face and the weight information of each key point based on the detection result of the face.
In this way, the temperature measurement may focus on positions that are not covered or affected by objects with abnormal temperature, thus improving accuracy of the temperature measurement.
After that, return to
It should be understood that although the temperature information acquired at this time may be configured as the basis for calculating the temperature 150, there may still be errors due to influence of an environmental factor and other factors. Therefore, a functional relationship between a measurement temperature and an actual temperature at a location where the temperature sensing device and the image acquisition device are arranged may be created. For example, this functional relationship may be fitted by a least square method based on prior knowledge. Thus, the computing device 120 may acquire a measurement temperature of each key point, and determine an actual temperature of each key point based on the measurement temperature. Therefore, accuracy of the actual temperature determined by the computing device 120 is significantly improved.
At 308, the computing device 120 may determine a temperature of the target position at least based on the temperature information and the weight information of each key point. It may be understood that the target position described herein may be at least one of a face, eyes and hands (including fingerprints) of the object, and the object is not limited to a person. For example, after determining the temperature of the target position, the computing device 120 may compare the temperature with a temperature threshold, and alarm when the temperature is higher than the temperature threshold. Since temperatures of respective positions of a human body are different, when the face of the person is detected, a temperature threshold corresponding the face may be preset to be different from a temperature threshold corresponding to the hands of the person. In addition, when performing temperature detection on animals in a zoo or a feeding station, because temperatures of respective animals are different, different temperature thresholds may also be determined for different species of animals, thus realizing the temperature measurement and alarm of different animals.
In the above way, the present disclosure may improve the accuracy of the temperature measurement by recognizing the plurality of key points and explicitly determining the temperature at each key point and the weight information of each key point. In addition, since the disclosure may be applied to a scene of multi-pedestrian, or multi-animal, etc., without intervention of staff, time and labor cost of the temperature measurement may be reduced, and a risk of staff being infected during epidemic may be reduced.
In addition, the computing device 120 may also recognize the target position at least based on the key points and the weight information, and determine the object based on a recognition result. In some embodiments, the computing device 120 may recognize a face based on the key points and the weight information, and then determine identity information of the monitored pedestrian. In other embodiments, the computing device 120 may also determine species of monitored animals based on the key points and the weight information. Since the mechanism of weight is set, the position covered will not or rarely be used by the computing device 120 for recognition, thus reducing the possibility of false recognition of the CNN 140 in the computing device 120.
In addition, the present disclosure also provides a system 500 for measuring temperature. As illustrated in
In this way, non-contact temperature measurement at system-level may be realized, and the accuracy of the temperature measurement may be significantly improved under a premise of constant computing capability demand.
In some embodiments, the system 500 may be applied to a temperature measurement scene of multi-pedestrian. For example, the image acquisition module 510 in the system 500 may be applied to a subway or an entrance of a venue, to achieve real-time acquisition of images such as RGB images and infrared images of pedestrians. The computing module 520 may perform image processing such as of the process 300 on the RGB images and the infrared images to obtain the temperature information of the pedestrians. Once it is found that a temperature of a pedestrian is higher than a predetermined threshold, the output display module 530 may lock the pedestrian through a variety of alarm methods, The system may monitor the temperature information of a plurality of pedestrians passing through the entrance in real time. In this way, direct contact between security inspection and epidemic prevention personnel and suspected patients may be avoided or reduced, and the process of the temperature measurement is simple and efficient, without causing congestion artificially.
In some embodiments, the system 500 may be applied to a farm or a zoo. For example, the image acquisition module 510 in the system 500 may be applied to a best viewing angle of the farm or zoo to achieve real-time monitoring of temperature information of animals. In addition, the computing module 520 may recognize species of animals, to determine the species of animals that have measured temperature, and thus acquire a temperature threshold of such a species of animals. Once it is found that a temperature of an animal is higher than a threshold value, the output display module 530 may lock the animal through a variety of alarm methods for the convenience of staff to treat or handle the animal. In this way, direct contact between the staff and the animals that may carry germs may be avoided or reduced.
In some embodiments, the key point information determination module 604 may include: a detection result applying module, configured to determine the key points and the weight information by applying the detection result to a key point determination model, wherein the key point determination model is obtained by training based on a reference target position in a reference image and reference key points of the reference target position and reference weight information.
In some embodiments, the temperature information acquisition module 606 may include: a temperature sensing image acquisition module, configured to acquire a temperature sensing image for the target position; and a temperature information determination module, configured to determine temperature information corresponding to a location of each key point from the temperature sensing image.
In some embodiments, the temperature information acquisition module 606 may include: a measurement temperature acquisition module, configured to acquire a measurement temperature of each key point; and an actual temperature acquisition module, configured to determine an actual temperature of each key point based on the measurement temperature.
In some embodiments, the apparatus 600 may further include: a target position recognition module, configured to recognize the target position at least based on the key points and the weight information; and an object determination module, configured to determine the object based on a recognition result.
In some embodiments, the target position is at least one of a face, eyes and fingerprints of the object.
In some embodiments, the target position detection module 602 may include: a detection region determination module, configured to determine a detection region for the target position in the input image through a detection region generation model.
Multiple components in the device 700 are connected to the I/O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; the storage unit 708, such as a disk, a CD, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via computer networks such as the Internet and/or various telecommunications networks.
The processing unit 701 executes the various methods and procedures described above, such as the method 200. For example, in some embodiments, the method 200 may be implemented as computer software programs, which are physically contained in a machine-readable medium, such as the storage unit 708. In some embodiments, some or all of the computer programs may be loaded and/or installed on the device 700 via the ROM 702 and/or the communication unit 709. The computer programs may execute one or more acts or steps of the method 200 described above when loaded to the RAM 703 and executed by the CPU 701. Alternatively, in other embodiments, the CPU 701 may be configured to execute the method 200 by any other appropriate ways (such as, by means of a firmware).
The above functions described herein may be executed at least partially by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components, including a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on chip (SOC), a load programmable logic device (CPLD) and so on, may be used.
The program codes for implementing the method of embodiments of the present disclosure may be written in any combination of one or more program languages. These program codes may be provided for a processor or a controller of a general-purpose computer, a special-purpose computer, or other programmable data-processing devices, such that the functions/operations regulated in the flow charts and/or block charts are implemented when the program codes are executed by the processor or the controller. The program codes may be completely executed on the machine, partly executed on the machine, partly executed on the machine as a standalone package and partly executed on a remote machine or completely executed on a remote machine or a server.
In the context of the present disclosure, the machine readable medium may be a tangible medium, which may include or store the programs for use of an instruction execution system, apparatus or device or for use in conjunction with the instruction execution system, apparatus or device. The machine readable medium may be a machine readable signal medium or a machine readable storage medium. The machine readable medium may include but not limited to electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses or devices, or any appropriate combination of the foregoing contents. A more detailed example of the machine readable storage medium includes electrical connections based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read only memory (an EPROM or a flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the above contents.
In addition, although respective act or step is described in a particular sequence, it should be understood that such act or step are required to be executed in the specified or sequential order as illustrated, or all illustrated acts or steps are required to be executed to achieve a desired result. Under certain environment, multitasking and parallel processing may be beneficial. In the same way, although several specific implementation details are included in the above discussion, these should not be interpreted as limitations of the scope of the present disclosure. Certain features described in the context of a single embodiment may also be in a combination manner to be implemented in a single implementation. On the contrary, the various features described in the context of a single implementation may also be implemented in multiple implementations individually or in any appropriate sub-combination.
Although language specific to structural features and/or method logic actions has been employed to describe the embodiments of the present disclosure, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features or acts described above are merely an exemplary form for implementing the claims.
Number | Date | Country | Kind |
---|---|---|---|
202010415405.7 | May 2020 | CN | national |
This application is the U.S. national phase application based on International Application No. PCT/CN2020/120964, filed on Oct. 14, 2020, which claims priority from Chinese Patent Application No. 202010415405.7, filed on May 5, 2020, the entire contents of which are incorporated herein by reference for all purposes.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CN2020/120964 | 10/14/2020 | WO |