Patent Application
20210325251
This Nonprovisional application claims priority under 35 U.S.C. § 119 on Patent Application No. 2020-074317 filed in Japan on Apr. 17, 2020, the entire contents of which are hereby incorporated by reference.
The present invention relates to an image display apparatus, an image display method, and a storage medium.
Patent Literature 1 discloses a pipe clogging diagnosis method by which it is possible to know the degree to which a pipe of a slurry transport line is clogged. The diagnosis method of Patent Literature 1 involves: capturing, with an infrared camera, images that represent transient changes in temperature of the external surface of a slurry pipe; and diagnosing the clogging of the slurry pipe based on thermal images obtained by processing the captured images.
[Patent Literature 1]
Japanese Patent Application Publication, Tokukai, No. 2013-83666
However, an area in which the diagnosis for clogging of the slurry pipe is carried out by the diagnosis method of Patent Literature 1 (hereinafter “diagnosis area”) is only an area where a temperature sensor such as the infrared camera is provided. Therefore, a user cannot carry out diagnosis in an area other than the area where the temperature sensor is provided.
Furthermore, according to the diagnosis method of Patent Literature 1, if there is an object that greatly differs in temperature from the diagnosis area in the vicinity of the diagnosis area, the temperature range of a thermal image, containing the diagnosis area and the object in the vicinity of the diagnosis area, is set wide to include the temperature of the object. If this is the case, the thermal image does not display slight temperature differences in the diagnosis area, making it impossible to carry out diagnosis in the diagnosis area on the basis of the slight temperature differences in the diagnosis area.
An object of an aspect of the present invention is to provide an image display apparatus, an image display method, and a storage medium each of which makes it possible to display a thermal image containing a desired area of a pipe or the like and each of which makes it possible, even if there is a great difference in temperature between the desired area and its surrounding area, to display slight temperature differences in the desired area in the thermal image.
In order to attain the above object, an image display apparatus in accordance with Aspect 1 of the present invention is an image display apparatus which is portable, including: a display section configured to display a thermal image in which temperature distribution on a subject is represented by colors, the subject including a pipe or the like for a fluid to flow therein; and a control section configured to control the display section, the control section being configured to control the display section to display the thermal image such that the temperature distribution on the subject is represented by colors within a temperature range having upper and lower limits that are settable.
An aspect of the present invention makes it possible to display a thermal image containing a desired area of a pipe or the like and, even if there is a great difference in temperature between the desired area and its surrounding area, possible to display slight temperature differences in the desired area in the thermal image.
(1-1. Overview)
The visible-light camera 17 is a camera that is sensitive to wavelength(s) in the visible wavelength range and that captures a visible light image by detecting visible light from a subject. The infrared camera 19 is a camera that is sensitive to wavelength(s) in the infrared wavelength range and that captures an infrared image by detecting infrared light from a subject. The field-of-view of the visible-light camera 17 and the field-of-view of the infrared camera 19 substantially coincide with each other. In the example illustrated in
The surrounding environment is, for example, an environment in which a support member supports the duct D1 and/or an environment in which an apparatus is disposed in the vicinity of the duct D1 when viewed from the image display apparatus 10. In this case, the subject also includes such a surrounding environment.
In the example illustrated in
In the example illustrated in
(1-2. Configuration)
It is noted here that, in Embodiment 1, the visible light image captured by the visible-light camera 17 is not essential to the image display apparatus 10. In Embodiment 1, the image display apparatus 10 does not necessarily need to use the visible light image captured by the visible-light camera 17. In Embodiment 1, a configuration in which the image display apparatus 10 includes the infrared camera 19 but does not include the visible-light camera 17 may be employed.
The display section 13 is composed of, for example, a liquid crystal display or an organic light emitting display. The touchscreen 15 is, for example, an input device that detects a user's finger or a stylus pen touching the touchscreen 15. The touchscreen 15 is disposed such that its area for operation is superimposed on a display area of the display section 13. The screen 10a in
The communication section 21 is a device for connection with a network (not illustrated). The communication section 21 communicates with the network in accordance with a communication standard such as 3G, 4G, or LTE. The interface section 23 is a device for connection with an external apparatus (not illustrated). The interface section 23 carries out communication in accordance with a communication standard such as USB (registered trademark), HDMI (registered trademark), or Bluetooth (registered trademark).
The storage section 16 is a storage medium that stores parameters, data, control programs, and/or the like for use in implementing a predetermined function. The storage section 16 is composed of, for example, a hard disk, a semiconductor storage, or a semiconductor memory.
The storage section 16 stores an image display program 16a and temperature range data 16b which are for implementing a function of the image display apparatus 10 (such a function will be described later). The image display program 16a is an example of a control program. The temperature range data 16b is used to set the upper and lower limits of a temperature range for use in generating a thermal image from an infrared image captured by the infrared camera 19.
The control section 11 includes a CPU. The control section 11 executes the image display program 16a to implement the function of the image display apparatus 10 (the function will be described later). Note that the control section 11 may be realized only by a hardware circuit designed especially for implementation of a specific function. The control section 11 may also include various circuits such as MPU, GPU, DSP, FPGA, ASIC, and/or the like in addition to the CPU.
(1-3. Operation)
As described earlier, in Embodiment 1, the visible light image captured by the visible-light camera 17 is not essential to the image display apparatus 10. The following description will first discuss Embodiment 1 based on an example configuration in which the image display apparatus 10 does not use the visible light image captured by the visible-light camera 17, and then discuss Embodiment 1 based on an example configuration in which the image display apparatus 10 uses the visible light image captured by the visible-light camera 17.
The following description will discuss operation in the case of the example illustrated in
The image display apparatus 10 has the function of causing the display section 13 to display a thermal image based on an infrared image captured by the infrared camera 19. The “thermal image” herein means an image which is generated from an infrared image and in which each pixel is colored according to temperature information indicative of the temperature of a part of a subject included in that pixel. The color of each pixel is set within a temperature range having upper and lower limits set. This makes it possible to generate a thermal image in which temperature distribution on the subject is represented by colors within the foregoing temperature range. This function of displaying an image is realized by the control section 11 executing the image display program 16a.
Note that the color of each pixel may be set such that at least one of the three components of color (i.e., “hue”, “saturation”, and “lightness”) varies depending on the temperature of that pixel. For example, the “saturation” and “lightness” may be varied with the “hue” fixed. Alternatively, the “hue” may be varied with the “saturation” and “lightness” fixed. The following description will discuss an example embodiment in which the “hue” is varied with the “saturation” and “lightness” fixed.
It is noted here that, in the example shown in
The screen example 70 in
The screen example 70 in
The following description will discuss how the image display apparatus 10 operates, with reference to the screen example 70 of
Step S101:
The control section 11 acquires an infrared image 19a of a duct D7 and its surrounding environment, captured by the infrared camera 19. The control section 11 causes the storage section 16 to store the infrared image 19a acquired from the infrared camera 19.
Step S102:
The first image processing section 11a of the control section 11 reads the infrared image 19a from the storage section 16. The first image processing section 11a acquires temperature information on each pixel included in the read infrared image 19a. The first image processing section 11a sets a reference temperature, with use of the temperatures of pixels which are included in the infrared image 19a and which correspond to the reference position mark 72 and its surrounding area. In a case where the temperatures of the pixels are different, the average of the temperatures of the pixels may be used as the reference temperature.
The first image processing section 11a sets the upper and lower limits of the foregoing temperature range with use of the set reference temperature and pre-set range values. In the screen example 70 of
The first image processing section 11a sets the upper and lower limits of the temperature range using the following equations:
Upper limit=17.5° C.+5° C.=22.5° C., and
Lower limit=17.5° C.−5° C.=12.5° C.
The first image processing section 11a allocates hue to each temperature included in the temperature range having the upper and lower limits set. The first image processing section 11a generates an image of the color bar 71a which reflects the hues allocated to respective temperatures. The image of the color bar 71a is, for example, an image that changes in color in the order of “blue”, “green”, and “red” with increasing distance from the lower limit and with decreasing distance to the upper limit. The first image processing section 11a refers to the temperature range data 16b stored in the storage section 16 to decide which hue is to be allocated to each temperature. The hues allocated to respective temperatures are, for example, stored as part of the temperature range data 16b in the storage section 16. The hues allocated to respective temperatures can be rewritten by a user anytime.
Note that, as shown in
Furthermore, as shown in
Note that the positions of the display areas 71b, 71c, 71d, 72a, 73b, and 73c shown in
The first image processing section 11a generates images that are used to display the color bar 71a, the reference position mark 72, the slide bar 73a, and the display areas 71b, 71c, 71d, 72a, 73b, and 73c on the screen 10a, respectively.
Step S103:
The second image processing section 11b of the control section 11 sets a reference temperature and upper and lower limits. A process of setting these is the same as in the case of the first image processing section 11a, and therefore descriptions therefor are not repeated here.
The second image processing section 11b reads the infrared image 19a from the storage section 16. The second image processing section 11b generates, from the read infrared image 19a, a thermal image in which temperature distribution on the duct D7 and its surrounding environment is represented by colors within the temperature range having the upper and lower limits set. Specifically, the second image processing section 11b refers to the temperature range data 16b stored in the storage section 16 to decide the color of each pixel of the infrared image 19a on the basis of the temperature of that pixel. The second image processing section 11b generates a thermal image in which each pixel is colored according to the temperature of that pixel.
In the example shown in
Step S104:
The first image processing section 11a outputs, to the display processing section 11d, the images that are used to display the color bar 71a, the reference position mark 72, the slide bar 73a, and the display areas 71b, 71c, 71d, 72a, 73b, and 73c on the screen 10a, respectively. The second image processing section 11b outputs the thermal image to the display processing section 11d.
The display processing section 11d causes the screen 10a to display the images inputted from the first image processing section 11a and the second image processing section 11b such that the images from the first image processing section 11a are superimposed on the thermal image from the second image processing section 11b.
Step S105:
The image display apparatus 10 stops operating if no user operation has been received via the touchscreen 15 (NO in step S105).
On the contrary, if a user operation has been received via the touchscreen 15 (YES in step S105), the control section 11 carries out step S102 again.
Specifically, as shown in
Note that the user can change only one of the current upper and lower range values by sliding only one of the sliders on the slide bar 73a with his/her finger or a stylus pen along the direction indicated by arrow A21 or along the direction indicated by arrow A22.
Then, the first image processing section 11a and the second image processing section 11b set the upper and lower limits of the temperature range on the basis of the upper and lower range values at least one of which has been changed. Then, the foregoing steps S102 to S105 are repeated.
The user can change the foregoing upper and lower limits by a simple action while checking the thermal image displayed on the screen 10a. Since a thermal image with a temperature range having upper and lower limits freely set by the user is displayed, a user-friendly method is provided.
A typical infrared camera automatically recognizes the highest and lowest temperatures in the captured image. The infrared camera allocates a hue to each temperature included in the temperature range defined by the recognized highest and lowest temperatures. For example, if there is an object that differs greatly in temperature from its surroundings in the captured image, the infrared camera automatically recognizes the temperature of the object as the lowest or highest temperature. Therefore, in an area that the user wants to examine, hue varies in large steps, and therefore it is impossible to represent slight differences in temperature. The user cannot recognize the slight differences in temperature from such variations in hue.
With the image display apparatus 10, in a case where there is an object that differs greatly in temperature from its surroundings in the captured image, the user can change the foregoing upper and lower limits by a simple action so that the temperature of the object is out of the temperature range. With this, the user can cause a slight temperature difference, in the area that the user wants to examine, to be displayed in a thermal image with a temperature range having the changed upper and lower limits.
The following description will discuss Embodiment 1 based on an example configuration in which the image display apparatus 10 uses the visible light image captured by the visible-light camera 17.
The following description will discuss operation in the case of the example illustrated in
The image display apparatus 10 has the function of causing the display section 13 to display (i) a thermal image based on an infrared image captured by the infrared camera 19 and (ii) a visible light image captured by the visible-light camera 17 such that the visible light image is superimposed on the thermal image. This function of displaying images is realized by the control section 11 executing the image display program 16a.
The following description will discuss how the image display apparatus 10 operates, with reference to the screen example 70 of
Step S201:
The control section 11 acquires a visible light image 17a of a duct D7 and its surrounding environment, captured by the visible-light camera 17. The control section 11 further acquires an infrared image 19a of the duct D7 and its surrounding environment, captured by the infrared camera 19. The control section 11 causes the storage section 16 to store the visible light image 17a acquired from the visible-light camera 17 and the infrared image 19a acquired from the infrared camera 19.
Step S202:
Step S202 is the same as step S102 of
Step S203:
Step S202 is the same as step S103 of
Step S204:
The first image processing section 11a outputs, to the display processing section 11d, the images that are used to display the color bar 71a, the reference position mark 72, the slide bar 73a, and the display areas 71b, 71c, 71d, 72a, 73b, and 73c on the screen 10a, respectively. The second image processing section 11b outputs the thermal image to the display processing section 11d. The third image processing section 11c reads the visible light image 17a from the storage section 16. The third image processing section 11c outputs the read visible light image 17a to the display processing section 11d.
The display processing section 11d causes the screen 10a to display the images inputted from the first image processing section 11a, the second image processing section 11b, and the third image processing section 11c such that the images from the first image processing section 11a are superimposed on the thermal image from the second image processing section 11b and the visible light image from the third image processing section 11c. In the screen example 70 of
Step S205:
Step S205 is the same as step S105 of
The following description will discuss Embodiment 2 of the present invention. For convenience of description, members having functions identical to those described in Embodiment 1 are assigned identical referential numerals, and their descriptions are not repeated here.
Embodiment 2 is different from Embodiment 1 in that the foregoing upper and lower limits are automatically set on the basis of the surface temperature of a duct and the temperature of an environment surrounding the duct, instead of setting the upper and lower limits on the basis of the reference temperature and range values. The following description will discuss this difference.
The inventors of the present invention have found that the foregoing upper and lower limits can be set on the basis of the relationship between the surface temperature of the duct and the temperature of the environment surrounding the duct (hereinafter may be referred to as “ambient temperature”). This relationship is discussed with reference to
Note that the duct D9 was made of iron, and that the deposit X9 was a deposit of iron fume. Also note that the measurement was carried out under the conditions in which the ambient temperature was 18° C. and under the conditions in which the ambient temperature was 30° C., while the fluid temperature was varied within the range of 40° C. to 80° C.
The results of the measurement shown in
The following conclusion was derived from the above-stated two findings.
In Embodiment 2, the storage section 16 pre-stores an experimental fact indicative of the degree of temperature distribution which has resulted from a certain surface temperature of the duct D9 and a certain temperature of the environment surrounding the duct D9. The experimental fact is an example of “relationship information”. In the example shown in
The screen example 120 in
The screen example 120 in
Furthermore, as shown in
Furthermore, as shown in
Note that the positions of the display areas 121b, 121c, 121d, 122a, and 123 shown in
Also in Embodiment 2, range values may be pre-set similarly to Embodiment 1. In this case, in the example shown in
[Software Implementation Example]
The control section 11 of the image display apparatus 10 can be realized by a logic circuit (hardware) provided in an integrated circuit (IC chip) or the like or can be alternatively realized by software.
In the latter case, the control section 11 includes a computer that executes instructions of a program that is software realizing the foregoing functions. The computer, for example, includes at least one processor and at least one computer-readable storage medium storing the program. An object of the present invention can be achieved by the processor of the computer reading and executing the program stored in the storage medium. Examples of the processor encompass a central processing unit (CPU). Examples of the storage medium encompass a “non-transitory tangible medium” such as a read only memory (ROM), a tape, a disk, a card, a semiconductor memory, and a programmable logic circuit. The computer may further include a random access memory (RAM) or the like in which the program is loaded. Further, the program may be supplied to or made available to the computer via any transmission medium (such as a communication network and a broadcast wave) which allows the program to be transmitted. Note that an aspect of the present invention can also be achieved in the form of a computer data signal in which the program is embodied via electronic transmission and which is embedded in a carrier wave.
Aspects of the present invention can also be expressed as follows:
An image display apparatus in accordance with Aspect 1 of the present invention is an image display apparatus which is portable, including: a display section configured to display a thermal image in which temperature distribution on a subject is represented by colors, the subject including a pipe or the like for a fluid to flow therein; and a control section configured to control the display section, the control section being configured to control the display section to display the thermal image such that the temperature distribution on the subject is represented by colors within a temperature range having upper and lower limits that are settable.
With the configuration, a user can carry around the image display apparatus, and therefore it is possible to cause the image display apparatus to display a thermal image including a desired area of the pipe or the like.
Furthermore, with the configuration, even if there is a great difference in temperature between the desired area and an area surrounding the desired area, is it possible, by setting the upper and lower limits of the temperature range, to display slight temperature differences in the desired area in the thermal image.
In Aspect 2 of the present invention, an image display apparatus in accordance with Aspect 1 is arranged such that the control section is configured to control the display section to display the thermal image such that the temperature distribution on the subject is represented by variations in at least one of (i) hue, (ii) saturation, and (iii) lightness.
With the configuration, it is possible to present, to a user, the temperature distribution on the subject represented by variations in at least one of (i) hue, (ii) saturation, and (iii) lightness.
In Aspect 3 of the present invention, an image display apparatus in accordance with Aspect 1 or 2 is arranged such that the control section is configured to control the display section such that, when at least one of the upper and lower limits of the temperature range has been changed, the display section displays the thermal image such that the temperature distribution on the subject is represented by colors within the temperature range in which the at least one of the upper and lower limits has been changed.
With the configuration, it is possible to present, to a user, the temperature distribution on the subject represented by colors within the changed temperature range.
In Aspect 4 of the present invention, an image display apparatus in accordance with any of Aspects 1 to 3 is arranged such that: the display section includes a screen in which the thermal image is displayed; and the control section is configured to set the upper and lower limits of the temperature range using, as a reference temperature, a temperature which is obtained from the thermal image and which is indicated at a predetermined position on the screen.
With the configuration, it is possible to set the upper and lower limits on the basis of the temperature indicated at a predetermined position on the screen.
In Aspect 5 of the present invention, an image display apparatus in accordance with Aspect 4 is arranged such that: the subject further includes an environment which surrounds the pipe or the like; and the control section is configured to set the upper and lower limits of the temperature range in accordance with relationship information pre-stored in the image display apparatus, the relationship information being indicative of a relationship between a surface temperature of the pipe or the like and a temperature of the environment.
With the configuration, it is possible to set the upper and lower limits on the basis of the surface temperature of the pipe or the like and the temperature of the environment.
In Aspect 6 of the present invention, an image display apparatus in accordance with any of Aspects 1 to 5 is arranged such that: the control section is configured to control the display section to further display a slide bar such that the slide bar is superimposed on the thermal image, the slide bar being a user interface via which the upper and lower limits of the temperature range are set; (i) an amount by which the upper limit of the temperature range is to be changed is decided in accordance with an amount by which a first slider on the slide bar has been slid and (ii) an amount by which the lower limit of the temperature range is to be changed is decided in accordance with an amount by which a second slider on the slide bar has been slid; and (a) whether the upper limit of the temperature range is to be increased or reduced is decided in accordance with a direction in which the first slider on the slide bar has been slid and (b) whether the lower limit of the temperature range is to be increased or reduced is decided in accordance with a direction in which the second slider on the slide bar has been slid.
With the configuration, a user can change, with intuitive operation, the amounts by which the upper and lower limits are to be changed and whether the upper and lower limits are to be increased or reduced.
In Aspect 7 of the present invention, an image display apparatus in accordance with Aspect 5 further includes a temperature sensor configured to measure the temperature of the environment.
In Aspect 8 of the present invention, an image display apparatus in accordance with Aspect 5 is arranged such that the control section is configured to use, as the temperature of the environment, an average of temperatures obtained from a part of the thermal image which part corresponds to the environment.
In Aspect 9 of the present invention, an image display apparatus in accordance with any of Aspects 1 to 8 is arranged such that: the display section is configured to further display a visible light image of the subject; and the control section is configured to control the display section to display the visible light image such that the visible light image is superimposed on the thermal image.
With the configuration, the visible light image and the thermal image are superimposed; therefore, a user can easily determine the position of the pipe or the like.
An image display method in accordance with Aspect 10 of the present invention is a method of causing an image display apparatus to display a thermal image in which temperature distribution on a subject is represented by colors, the subject including a pipe or the like for a fluid to flow therein, the image display apparatus being portable, the method including the steps of: acquiring an infrared image of the subject captured by an infrared camera; generating, from the infrared image, the thermal image such that the temperature distribution on the subject is represented by colors within a temperature range having upper and lower limits which have been set; and causing the image display apparatus to display the thermal image.
An image display apparatus (in particular, a control section) in accordance with each aspect of the present invention can be realized by a computer. The computer is operated based on (i) a control program for causing the computer to realize the image display apparatus by causing the computer to operate as each section included in the image display apparatus and (ii) a computer-readable storage medium in which the control program is stored. Such a control program and a computer-readable storage medium are included in the scope of the present invention. An image display apparatus in accordance with each aspect of the present invention may be realized as an integrated circuit (IC chip). A chip including the integrated circuit, and the like, are also included in the scope of the present invention.
The present invention is not limited to the embodiments, but can be altered by a skilled person in the art within the scope of the claims. The present invention also encompasses, in its technical scope, any embodiment derived by combining technical means disclosed in differing embodiments.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2020-074317 | Apr 2020 | JP | national |
