Patent Application
20150116482
The present invention generally relates to industrial plants. More particularly the present invention relates to a method, inspecting device and computer program product for enabling a user in an industrial plant to inspect an object of the plant.
In the facilities of industrial plants there may occur situations where maintenance engineers need to look at activities occurring at objects, such as plant equipment.
When for instance conducting daily morning inspection rounds at a plant, plant maintenance engineers may need to check the status of plant equipment. User studies show that while doing these rounds, the engineers use paper and a pen to write down remarks about possible problem areas and needed fixes. Afterwards when the plant maintenance engineers return to their office location, they have added these notes to a computer system by use of a desktop computer.
In many plants mobile devices and digital cameras have been recently introduced to assist the plant maintenance engineers in their daily work tasks. The engineers now take photos of problem areas or issues by using the camera. These photos together with notes taken may later be shared with colleagues or documented.
The photos may also later get compared with earlier photos taken of the equipment if the plant maintenance engineers want to follow a progress of equipment to figure out if there is a potential problem occurring in the future. In the user studies, one such scenario was to follow a water leakage at a plant by taking photos of the area during several days.
There is however a problem in locating the correct object for the comparison. There may for instance exist a great number of the same type of objects at different locations in the plant.
Even if the correct object is located it is not certain that a good comparison can be made, since the object may be captured from different angles that may not be simple to compare.
There have been developed some useful ideas in the field of consumer devices.
US 2011/0128395 does for instance describe how a user of a mobile terminal can choose to see differences between a reference image and a preview image, then compare the reference image with the preview image and display the comparison result. The document also mentions that the preview image may be overlaid on the reference image. The document further describes how the preview image is displayed on a display and then captured to generate a comparison image. The examples of use given are all related to portrait images.
WO 2011/071861 describes matching of a captured image with a reference image using location information. In this document an image is first captured and then a reference image is fetched for being compared. The document describes the performing of these types of activities in relation to objects of interest such as buildings or monuments. It would therefore seem that the function at hand has been developed for a tourist.
There is thus still a need for improvement when comparing images of objects in industrial plants.
The present invention addresses one or more of the above mentioned problems.
The present invention addresses this situation. The invention is therefore directed towards solving the problem of providing an improvement in the comparison of images of objects in industrial plants.
This object is according to a first aspect of the invention solved through a method of enabling a user in an industrial plant to inspect an object of the plant, where the user is equipped with a mobile terminal, the method being performed by an inspecting device and comprising the steps of:
This object is according to a second aspect of the invention solved through an inspecting device enabling a user in a plant to inspect an object of the plant, the inspecting device being provided in a mobile terminal equipped with a viewfinder and being configured to
guide the user to the location of the object in the plant, and
present a reference image of the object in the viewfinder of the mobile terminal when the location of the object has been reached in order to allow the user to capture an image of the object.
This object is according to a third aspect of the invention solved through a computer program product computer program product enabling a user in an industrial plant to inspect an object of the plant, in which plant users are being equipped with mobile terminals, the computer program product being provided on a data carrier comprising computer program code configured to cause an inspecting device provided in a mobile terminal equipped with a viewfinder to, when the computer program code is loaded into the inspecting device,
The present invention has a number of advantages. It allows the correct object to be located for the comparison of images. This is especially important if different engineers are involved in the inspecting of the object. Through providing the reference image in the view-finder it is ensured that the angle used for the reference image is easily obtained also for the newly captured image, thereby ensuring a good comparison. Plant maintenance engineers can also perform preventive maintenance through more effectively following the status of the object to find out possible problems. The images can be used for sharing information between engineers and the progress of a possible issue can be followed by many users, not only one.
The present invention will in the following be described with reference being made to the accompanying drawings, where
In the following, a detailed description of preferred embodiments of a method, inspecting device and computer program product for enabling a user in an industrial plant to inspect an object of the plant will be given.
In
There is furthermore a second data bus B2 and between the first and second data busses B1 and B2 there are connected a server 18 providing control and protection of the process and a database 20 where data relating to control and protection of the process is stored. Such data relating to control and protection may here comprise process control data such as measurements and control commands, while data relating to protection may comprise alarm and event data as well as data on which alarms and events can be generated, such as measurements made in the process. There is furthermore a guiding server 22 connected between the two buses B1 and B2. This guiding server 22 may in one variation of the invention provide guiding instructions used to guide a mobile terminal. In some variations of the invention the server may also transfer a reference image to a mobile terminal. How this may be done will be described in more detail later. In other variations of the invention the guiding server is not needed. Hence, it may be omitted.
To the second data bus B2 there is furthermore connected a number of further devices 24, 26, 28 and 30. These further devices are often termed field devices forming interfaces to the process. A field device is thus an interface via which measurements of the process are being made and to which control commands are given. One or more of these field devices may furthermore be able to generate alarms. A first field device 24 is as an example an object that needs surveillance and is here exemplified by a tank. The invention will in the following be described in relation to this object. It should here furthermore be realized that such an object needing surveillance is not necessarily a field device, i.e. a device directly involved in the control of the process. It may thus be another type object in the plant needing surveillance, for instance a device being merely associated with the process.
Furthermore the positions of the wireless access points 46, 48 and 50 are typically known and because of this also the positions of the first mobile terminal and consequently the user of this terminal may be known.
A first embodiment of the invention will now be described with reference also being made to
The users of the mobile terminals 32, 34, 36 and 38 may typically be plant maintenance engineers performing some activity in the premises, such as inspecting plant equipment.
The user of the first mobile terminal 32 may be such a maintenance engineer.
When conducting daily morning inspection rounds at the plant, the plant maintenance engineers may check the status of plant equipment, such as the object that in this example is the first field device 24. User studies show that while doing these rounds, the engineers have traditionally used paper and a pen to write down remarks about possible problem areas and needed fixes. Afterwards when the plant maintenance engineers return to their office location, they have added these notes to the system by use of desktop computers, like the operator terminals 12 and 14.
However, in many plants mobile terminals and digital cameras have been recently introduced to assist the plant maintenance engineers in their daily work tasks. The engineers now take photos of problem areas or issues by using a camera. These photos together with notes taken are later shared with their colleagues or documented.
The photos may then be later compared with earlier photos taken of the equipment as the plant maintenance engineers want to follow a progress of equipment to figure out if there is a potential problem occurring in the future.
It may as one example be of interest to follow the progress of a water leakage at the object 24 of the plant by taking photos of the area during several days, where the object is a tank in this example.
There are a number of problems associated with the traditional way of handling this situation:
The present invention presents a tool to help plant maintenance engineers to, by use of their mobile terminals, repeatedly capture images of the same scene in a factory plant aiming to follow the progress of a possible issue.
In short, a mobile terminal will according to the invention be used to guide the engineer taking photos to the same spot and present a reference image of the object in the viewfinder in order to ensure that automatically or manual follow-up of the possible issue can be performed correctly.
The method of the first embodiment of the invention therefore starts by the inspecting device 41 of the first mobile terminal 32 receiving a reference image RI 52, step 58. This reference image 52 may be received from the guiding server 22 via the wireless network WN. The reference image 52 may therefore be received via the radio circuit 42 and antenna 44 in the first mobile terminal 32 and forwarded from the radio circuit 42 to the inspecting device 41. There may also be a position associated with this reference image 52, which is the position at which the reference image 52 was captured. Also data about this position may then be received together with the reference image 52. In the example the reference image 52 is an image of the object 24. It should here be realized that the reference image 52 may be received from another entity such as from another mobile terminal of another engineer. The reference image 52 may furthermore have been captured via the first mobile terminal 32, in which case the reference image 52 may already be provided in the first mobile terminal 32. It should therefore be understood that this receiving step is optional.
Therefore, in case the reference image 52 is captured using the first mobile terminal 32, then the plant maintenance engineer may have captured the reference image 52, which may be an image of a possible issue of the plant in relation to the object 24. It is also possible that the reference image 52 has been captured earlier by another engineer using the first mobile terminal 32.
However, in order to be able to use the reference image 52 the current user of the first mobile terminal 32 needs to be guided to the location of the object 24. Therefore the inspecting device 41 of the first mobile terminal 32 guides the user to the location of the object.
There are a number of ways in which this may be done. The guiding server 22 may for instance continuously keep track of the users of the mobile terminals 32, 34, 36 and 38 as well as their positions in the plant 10. Positions may be obtained via the wireless network WN. The positions of the mobile terminals 32, 34, 36 and 38 may more particularly be obtained through knowledge of which wireless access points 46, 48 and 50 they are in contact with. A mobile terminal may for instance be assumed to have the position of the access point with which it is connected. The signal strength of the communication between the mobile terminal and access point may also be used to determine the distance of a mobile station from the access point, which gives a radius around the access point at which the mobile terminal may be located. This together with knowledge of the layout of the premises, such as where walls, floors and ceilings are provided, may be used for estimating the position. Furthermore, if a mobile terminal is in contact with more access points, then triangulation may be used. Here the points of intersection of the radiuses of two or three access points may be used for determining the position. Also this may be combined with knowledge of the layout of the premises 45 in order to determine the position of the mobile station.
The inspecting device 41 may in this case receive instructions from the guiding server 22 for reaching the position of the object 24. As an alternative it may itself determine what instructions are to be given.
The guiding server 22 may thus provide guiding instructions for guiding the user to the location of the object 24. The guiding instructions for the user of the first mobile terminal 32 may for instance be based on a current location of this first mobile terminal 32 and comprise a number of instructions guiding the user from this current position to the location of the object 24.
The guiding server 22 may transmit all this data with the help of the wireless network WN and more particularly using the various access points of the wireless network WN with which the first mobile terminal 32 is communicating.
The first mobile terminal 32 may thus receive the guiding instructions using the radio circuit 42 and the antenna 44. The instructions are then forwarded from the radio circuit 42 to the processor 38 which implements the inspecting device 41. It is here as an alternative possible that the inspecting device 41 itself has a set of instructions for guiding the user to the location of the object and then it itself determines the current position and the instructions to be used for guiding to the location of the object. In case both the current position and object are located outside of any premises the first mobile terminal 32 may instead use GPS for guiding. If the first mobile terminal 32 is provided with an accelerometer and/or a gyro, these may as an alternative also be used to provide the position of the first mobile station 32.
It can in this way be seen that the inspecting device 41 guides the user to the location of the object 24 using the received or own generated guiding instructions, step 60. The guiding instructions may provide directions and may furthermore be dynamically updated based on the positions of the first mobile terminal 32.
The location at which the reference image 52 was captured may thus be tracked by using GPS or indoor positioning system such as Wi-Fi or Bluetooth. Also, the position of the mobile terminal 32 can be tracked using accelerometer and gyro.
When the location of the object 24 has been reached, and in this example when the first mobile terminal 32 has reached the first room comprising the object 24, the inspecting device 41 displays the reference image 52 in the viewfinder VF provided through the display of the touch screen 34, step 62. The reference image 52 is furthermore displayed together with the view that is observable through the view-finder. The reference image 52 is thus presented together with a view that may be captured by the engineer. The reference image may for this reason be displayed semi-transparently in the view-finder. As an alternative to semi-transparent presentation it is possible that only the contours of objects in the reference image are presented in the viewfinder.
In this way the reference image 52 of the object 24 may be used for investigating the progress of the previously-mentioned issue of the object 24, which was here exemplified by a water leakage.
Thus, in order to later follow the progress of the issue, the reference image 52 of the scene of the issue can be used as a reference when taking new photos of the same scene. Hence, the viewfinder VF of the mobile terminal camera will show the reference image 52 together with the observable view so that the engineer can take a new photo 54 in a similar way. Through showing the reference image 52 in the viewfinder VF, the user will be able to align the view-finder with the object so that a new image 54 can be captured at the same distance and angle as that used for capturing the reference image 52. The follow-up photo 54 can also be taken by another person, as he knows how to take the picture using the camera viewfinder containing the reference photo.
After the reference image 52 has been presented, the inspecting device 41 therefore waits for the user of the first mobile terminal 32 to capture an image. The user may for instance select to capture an image through touching a button on the touch screen 34.
If an image capturing instructions is received, step 64, which is a user selection of the capturing of an image, then an image CI 54 is captured, step 66, by a camera function provided by the inspecting device 41.
This image 54 can then be compared with the reference image 52 in order to determine how serious the issue is.
Imaging the following scenario:
1. The plant maintenance engineer is doing his daily round at the plant.
2. He notices that there is water leakage around one device 24.
3. He decides to follow the progress of the leakage and takes a photo 52 of the scene.
4. Next week, when he conducts the same round again, he is guided to the device 24 where he last week noticed the water leakage.
5. As he wants to take a photo to compare the leakage with the photo taken last week, he opens up the camera application, which shows the earlier photo transparently in the camera viewfinder.
6. The plant engineer is now able to take a photo of the scene from the same spot by focusing the picture 54 through the earlier taken photo 52 that is shown transparently in the viewfinder.
7. By comparing and examining the reference and captured image or if more than one image is captured by comparing and examining a series of pictures the engineer can conclude that the water leakage is indeed getting worse, therefore he has enough information to indicate it as a high priority maintenance task.
It is in the first embodiment possible that the inspecting device 41 is itself able to determine the severity of the issue. In the first embodiment this is done through the inspecting device 41 analyzing the difference between the reference image 52 and the captured image 54, step 68. This may involve determining the difference through an analysis of the difference in pixel by pixel. After this difference has been determined, the inspecting device 41 may diagnose the issue. Therefore the inspecting device 41 diagnoses a condition of the object based on the analysis, i.e. it diagnoses the issue based on an analysis of the difference between the images. The difference may for instance indicate that the leakage is increased or decreased, which may be used for diagnosing how severe the leakage is.
It may optionally be possible that the results are to be transmitted to another device, such as to another mobile terminal or to the guiding server.
Therefore the inspecting device 41 may await, via the touch screen 34, a decision by the user to transfer the results to another device.
If such a decision is received, step 72, then the inspecting device 41 goes on and transmits the results. It may here transmit the captured image 54 and the reference image 52, step 74. It may also transmit the diagnosis, step 76. The transmitting may typically be done through the use of the radio circuit 42 and antenna 44. It should here be realized that in one variation, the only thing that is transmitted is the captured image 54.
After the results have been transmitted, the method is ended, step 78, which is also the case if the user would not select to capture an image, step 64, or not select to transfer results, step 72.
The above-described invention has a number of advantages. It allows the correct object to be located for the comparison of images. This is especially important if different engineers are involved in the inspecting of the object. Through providing the reference image in the view-finder it is ensured that the angle used for the reference image is easily obtained also for the newly captured image, thereby ensuring a good comparison.
There are several further benefits associated with the invention:
The investigating device may, as was previously described, be provided in the form of one or more processors together with computer program memory including computer program code for performing it's function. As an alternative it may be provided in the form of a Application Specific Integrated Circuit (ASIC) or Field-Programmable Gate Array (FPGA). This computer program code may also be provided on one or more data carriers which perform the functionality of the present invention when the program code thereon is being loaded into a mobile terminal. One such data carrier 80 with computer program code 82, in the form of a CD ROM disc, is schematically shown in
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/EP2012/060000 | 5/29/2012 | WO | 00 | 11/4/2014 |
