Patent Grant
8442800
The object of this invention is a method and a system for tracking objects that uses a dense sensor field.
In prior-art solutions the presence, location and movement of people, animals, objects and devices are detected using microwave radars or ultrasound radars, infrared detectors, video imaging and analysis of it, near-field sensors installed in the floor, pressure sensors or separate touch sensors used on top of the floor covering.
In prior-art solutions detection of an object is based on making an observation that covers a short period of time, or that is instantaneous, with detectors or sensors. The area to be monitored is covered with detectors or sensors, or a sensor or detector is disposed in a local object, and each observation produced by the sensor or detector that is stronger than a threshold value and exceeds a set limit for its duration is used to affirm an event of the object of interest. In some cases the condition for affirming an event can be an observation applying to a part of the sector of some detector or of a corresponding monitored area. Solutions in practice are e.g. the use of an infrared detector or a microwave radar for detecting a person arriving in a space or the use of a pressure sensor mat for detecting a patient getting out of bed. The problems of prior art solutions are the difficulty of making the correct interpretations that adequately contain the necessary information using sensor observations that are instantaneous or of very short duration. The unreliability and inaccuracy of information reduces its value. An example of the unreliability and inaccuracy of event information is alarm information applying to getting out of bed given by a sensor mat next to the bed in a situation in which another person has arrived next to the bed.
The problem of camera surveillance is that it typically requires that a person interprets the surveillance images in order for events that require actions, or that otherwise need detecting, to be detected. The automated interpretation of images requires expensive equipment, and often the interpretation of images anyway requires that a person makes an interpretation in order to achieve adequate accuracy of the content of the event information and reliability of the information.
Some solutions also present surveillance solutions wherein an RFID identifier is fixed to the moving objects in the space to be monitored. The problem with these solutions is that only those objects to which an identifier has been fixed are detected. An active identifier provided with a power source is used in some solutions, a problem in which is the duration of the power source, because typically there are very many excitation transmitters that activate the identifier in the space to be monitored and the identifier correspondingly activates many times.
The use of a dense near-field sensor field in detecting presence is presented in U.S. Pat. No. 6,407,556B1, among others.
The use of pressure sensors to detect presence or movement is presented in U.S. Pat. No. 4,888,581A1, among others.
There are many prior-art solutions for the tracking of a number of objects (multitarget tracking). The linking of the type of the object to the object as a part of the tracking of the object is presented in U.S. Pat. No. 6,278,401 B1, among others.
The use of a near-field sensor that is installed in the floor and measures an electrical connection for making observations is presented in application W02005020171A1, among others.
One problem of prior-art solutions that produce event information is the great need for processing power required to identify an object and the events linked to the object. For example, the identification of outlines on the basis of a video image can require e.g. hundreds of kilobytes per second of real-time analysis. On the other hand, one problem is also the relatively high amount of errors that occur in identification. More particularly, reliable identification of events linked to the tracked object by utilizing prior-art solutions has proven to consume resources and to be prone to operational error.
In prior-art solutions, the inflexibility of the solution is a problem when using detectors installed in locations, or targeted at them, as the object of the monitoring. When the use of the space to be monitored changes, and the event that is the object of the monitoring moves to a new location, or the event changes to another, the location and/or the targeting of the detectors must be changed. Changes require actions that during the lifecycle of the solution incur substantial modification costs and equipment costs, and also restrict the use of the spaces during the modification process.
The invention presents a system and a method, based on a dense sensor field, for tracking objects, which detects objects by tracking and events linked to the objects and to the space to be monitored by using predefined event conditions, and produces event information describing these events for use immediately or later.
The system according to the invention comprises a sensor field comprising two or more sensors in the vicinity of the object that are suited to the detective measurement of touch and/or pressure, measuring electronics that produces sensor observations by means of sensors, and a data processing apparatus, suited to processing sensor observations, comprising a processor and a memory. The system is characterized in that the data processing apparatus is arranged to detect the object to be tracked and to detect an event linked to the object on the basis of one or more sensor observations.
The processor of the system and the data processing apparatus, which comprises a memory, can be arranged to track an object by means of sensor observations.
The sensor observations used to detect or track an object and/or to identify an event linked to the object can be sequential in time.
The sensor field can comprise on average e.g. 4, 9 or 49 sensors per square meter. The strength of the sensor observations can vary e.g. according to the size, distance and/or material of the object causing the sensor observation.
The sensors of the system can be arranged to produce sensor observations e.g. at intervals of 0.01 or 0.1 seconds.
The sensor observation can be located e.g. on the basis of the location of the sensor that made the observation.
The system can detect an object e.g. on the basis of the strength of the observations and on the basis of their interpositioning. The system can track an object e.g. on the basis of a change in the location of the observations linked to it. In one preferred embodiment of the invention the data processing apparatus can process observations e.g. such that the system detects sensor observations linked to an object that are measured at different moments of time and collected for a period of at most five minutes by processing events linked to the object. An event linked to the object can be e.g. a change in status of the object, e.g. a movement in the sensor field in the space being monitored, an arrival into this space, an exit from the space, stopping or falling. A change in status of the object can be detected e.g. on the basis of the extent of the observations caused by the object, the shape of the outline formed in the sensor field from the observations and/or the strength of one or more sensor observations. Also the speed of the change of status of the object can be utilized in identifying an event.
The system can include an object in the tracking by recording at least one status information about the object, which status information describes the location or other possibly changing characteristic of the object at a certain moment of time. The system can estimate the probable new values of the status information of an object on the basis of the values recorded earlier and on the basis of the time that has passed from the moment in time they represent. In addition, tracking an object can comprise linking sensor observations to the object. The system can produce an association, which describes the linking of sensor observations to the tracked objects, and which is formed such that it describes how the tracked objects probably caused the sensor observations. The system can produce this association such that the system uses estimates, applying to the moment in time of the sensor observations, about the status information of the tracked objects and e.g. uses information about the estimated location of each object. The system can e.g. select the linking of an observation from the objects to be tracked to an object or to those objects which, on the basis of the status information, representing the moment of observation and estimated by the system, and on the basis of the estimate of the system applying to the creation of the observations, most probably influenced the observation in question. The system can update the status information of an object to be tracked by recording in it new or changed information on the basis of the observations linked to the object. Information applying to numerous moments of time can be recorded in the status information of an object. The content of the status information of an object can be e.g. the location, speed, direction, acceleration, size, shape, extent, density, way of moving of the object and/or other characteristic of the object inferred on the basis of observations. For example, the way of moving of an object can be recorded in the status information of the object on the basis of the shape of the outline formed by the observations linked to the object. For example, in a case in which the object to be tracked is a person, the way of moving can be recorded e.g. according to whether the sensor observations are suited to being caused by a person progressing by walking, running or crawling.
The system can process sensor observations such that it estimates the probability that the sensor observation of one or more certain moments of time are caused by an object that is not included in the tracking. The system can compare this probability to the probability with which the same observation is caused by an object included in the tracking. On the basis of the comparison and of the observations that are their basis of them, the system can include a new object detected in this way in the tracking.
The strength of the sensor observation or sensor observations linked to an object can be used to locate the position of the object or for inferring, recording and/or updating other information applying to the object.
The system can identify an event to be linked to an object on the basis of the sensor observations of one moment of time or of different moments of time, and/or on the basis of the information applying to one or more objects describing one moment of time or a number of moments of time. The system can use one or more event conditions known by the system for identifying an event. The system can compare information formed from sensor observations to an event condition or to event conditions in order to identify an event.
The system according to the invention can further comprise means for recording event conditions.
An event condition can comprise e.g. a condition or conditions applying to the presence, location, movement, shape, size or other information describing a characteristic, feature or status detectable with sensor observations or based on sensor observations. An event condition can also comprise a combination of conditions for the information describing more than one object. Furthermore, an event condition can comprise a combination of conditions for information describing a number of objects.
An event condition can be e.g. such that the individual conditions that it contains are fulfilled when the system compares them to a certain type of information recorded in the tracking of a person. An event condition can be e.g. such that its conditions are fulfilled when it is compared to information which is recorded e.g. when a person arrives in a space, changes walking to running, falls, gets out of bed, exits the space, changes to become undetectable with the sensors or when two people meet, a person picks an item or leaves his/her traces on an item. The content of an event condition can be e.g. a change in the essence of the object. An event condition can e.g. be such that it is fulfilled as a consequence of the types of observations that are produced when an item brought into the space starts to melt or to leak liquid. The content of an event condition can also be e.g. conditions applying to the location and speed of one object that are fulfilled when the speed of the object exceeds a given limit value when the object is located in a certain area. This kind of event condition is suited to detecting running in an area in which it is not permitted for reasons of safety.
The system can use a combination of conditions as an event condition, which comprises a number of conditions applying to the object. An event condition can be e.g. a combination of conditions applying to two objects, which is implemented if the speeds of and the distance between the objects fall below given limit values for at least a set length of time. This kind of event condition is suited e.g. for detecting money exchange or drug dealing in a space that is intended for passing through.
The system can further comprise means for identifying the type of a detected and/or tracked object by comparing the sensor observations and the information about the detected or tracked object to one or more identification profiles. An identification profile can comprise information about e.g. the area, number, strength of the sensor observations typically caused by an object or about the typical speed of movement of an object in the sensor field. The system can further comprise means for recording and reporting an object as an object of an unknown type. An object of an unknown type can be identified e.g. manually and information about its type can be recorded in the information of the object.
The system according to some embodiments of the invention can comprise ways and means for identifying an object with external means suited to the purpose, e.g. with an RFID reader. The system can receive information delivered by the external means e.g. about the identity and estimated location of some object, and the estimated point in time when the object was in this location. Further, the system can produce and deliver an identification request for implementing the tracking of some object, on the basis of the location of the object to be tracked and the known local coverage of the external means used to identify the object, to the aforementioned external means and receive information about the identity of the object delivered by the external means as a response to this request. The system can compare information received from the external means, and known information about the characteristics of an object on the basis of it, to the information recorded in tracking the objects and e.g., when the sets of location information match, the system can record the external identity of the object, and/or the type of the object known on the basis of it, in the information of a certain object to be tracked.
The system according to some embodiments of the invention can comprise one or more event conditions, which comprise a condition or conditions applying to the type or the identity of an object.
The system according to some embodiments of the invention can process sensor observations such that when processing the observations, and when tracking the objects, information is used that describes the delimitation of the space to be monitored with the sensor field, e.g. according to the doors, walls and corresponding factors, observation needs that differ from each other of the different areas of the space, or other factors that affect the use and observation need of the space and the furniture located in the space.
The system according to some embodiments of the invention can process sensor observations such that when processing the observations, and when tracking the objects, information is used when linking the sensor observations to a new object that describes the characteristics of the space to be monitored with the sensor field, which can increase or decrease the probability of the appearance of a new object compared to what it is elsewhere in the space to be monitored.
The system according to some embodiments of the invention can process sensor observations such that information for updating the information applying to the status of the objects to be tracked is used in the processing, which information describes the characteristics of the space to be monitored with the sensor field, which characteristics affect the supply of sensor observations about the objects. The system can use information applying to e.g. the location of furniture as this type of information. The system can e.g. deem that an object is recorded as being located in a certain shadow area until a new observation about the object is obtained as a result of the exiting from the shadow.
The system according to some embodiments of the invention can in tracking objects use information which describes areas that delimit the space to be monitored, which are closed from the standpoint of movement of the object, from where the object is not assumed to exit otherwise than by returning to the monitored space. The system can use information about e.g. a cupboard, bathroom or balcony as this type of information The system can e.g. record a person as being on a balcony as a result of the observations received about the person indicating that the person has moved to the balcony along a route leading there via a door opening.
The means that some embodiments of the system according to the invention comprise for processing sensor observations can comprise the information describing the location, size, position, movement components of the plane of the sensor field, distance from the plane of the sensor field, a certain physical characteristic of an object, other corresponding information, information about the speed of change of the status or of a characteristic of an object, or a combination of these information sets, being used as information applying to the status of the objects.
The means that some embodiments of the system according to the invention comprise for processing sensor observations can comprise the characteristics of an object being inferred on the basis of the sensor observations linked to an object. The characteristics that can be inferred can be e.g. the extent, shape, height, composition, distribution of mass, ability to move, or distribution probability of the object, that is projected to the sensor field.
The invention also relates to a method that can be implemented with the systems according to the different embodiments of the invention.
An advantage of the invention with respect to prior-art solutions is e.g. that with the method and the system it is possible to produce appropriate information about the events of the space to be monitored in a format that is well suited to the use of people and equipment. The method and the system identify events according to given event conditions with great accuracy such that event information with the correct content that describes an event is formed about exactly the desired events of the target space. The method and system according to the invention allow the detection and identification of events according to the defined use of each room such that event information is obtained about exactly those events about for information is needed. Furthermore, accuracy of the identification of events as well as correctness of the content of event information is achieved that are better than what can be achieved with economically comparable prior-art solutions. An advantage of the invention with respect to prior-art solutions is the high utility value of the observation information produced by processing the sensor observations and/or of the event information produced on the basis of it compared to the equipment resources required for producing and analyzing the information, e.g. to the amount of processing capacity or of memory.
An advantage of the method according to the invention is that by processing sensor observations measured at different moments of time, the necessary event information is produced with a lesser amount of sensors per unit of area than what is required in prior-art solutions to produce information that is just as accurate and reliable. The “resolution” of the sensor field can be set in different embodiments to be suited to the usage purpose. In the sensor field in some embodiments the extent of an individual sensor as well as the distance between sensors can also be arranged to be big, e.g. to tens of centimeters. In some other embodiments of the invention the distance between the sensors can be small, e.g. a few centimeters, in which case more observation data is obtained. The observation data produced by the sensor field can, in addition to the size of the object to be detected, also depend on its other properties, such as e.g. the material. The system can use a large amount of the observations applicable to the recent history of the object that is linked to the object by means of the tracking of the objects for detecting an event linked to an object. One advantage of this method compared to prior-art methods is that the observations obtainable at each moment of time do not necessarily need to be as accurate as when using a method that uses a short or instantaneous observation, which makes possible the use of a simpler sensor field and possibly one that is less expensive in terms of costs.
An advantage of the invention with respect to prior-art solutions is also better flexibility, because when the use of the space to be monitored changes, the physical system does not require changes. With a new placement of events that are an object of interest, e.g. when changing the locations of furniture or walls, the system can be adapted to the situation by changing the event conditions in a manner that corresponds to the change in the use of the spaces.
One possible advantage of the method and system according to the invention is that its technical simplicity and the economic inexpensiveness resulting from it makes possible an improvement in safety and operational efficiency by monitoring also the types of spaces the monitoring of which is not economically or technically reasonable with prior-art solutions, and by producing event information in these spaces about moving objects in different spaces that can be used for living, staying, production, leisure, retailing or other purposes.
Further, one advantage of the method and the system according to the invention can be that it detects and produces information about the falling of a person such that it is possible for the recipients of the information to quickly provide help to prevent and to mitigate the injuries of injured persons caused by falls. In the monitoring implemented with prior-art solutions the corresponding information is more unreliable, which reduces the utility value of the information owing to the cost, trouble and other inconvenience caused by “false alerts”.
Yet another advantage of the method and the system according to the invention with respect to prior-art solutions can be that the means for detecting the identity of an object used in connection with it needs to cover only certain points of the possible routes of objects. When an object passes via this type of point the system receives information about its identity, and when the object moves in the area to be monitored its identity is known by the system as a part of the information used in tracking the object.
An advantage of the system and method with respect to prior-art solutions is that determination of the identity of an object based on e.g. the reading of an RFID identifier can be connected to the system according to the invention such that the excitation used by the external means in the reading is sent according to the location of the object to be identified, which causes the activation of only the identifiers located in the desired area so that the received volume of responses sent by the identifiers is reduced and there is no need in the receiving arrangements to take into account responses about locating the identifiers sent, so that the arrangement can be implemented with a small amount of receiving apparatuses. Furthermore, other advantages include the fact that RFID reader collisions and RFID tag collisions are avoided, and no other solutions are needed for these.
In the following the invention will be described in more detail with reference to the embodiments presented as examples and to the attached drawings, wherein
In one preferred embodiment of the invention the extent of the observations that express the proximity of some body part by their strength that are linked to an object are used as an event condition of a falling event, expressed as the area covered by the observations and as the largest distance between the observations, as a change in the speed of the extent, and as the subsequent permanence of the location and strength. Observations, which change at a determined speed from observations corresponding to a vertical attitude to observations corresponding to a fallen person, are interpreted according to the condition as falling.
In one preferred embodiment of the invention the appearance conditions of different types of objects are used for detecting the appearance of new objects. The appearance conditions guide the operation of the system by setting sensor observations for each object type that interpret that an object has appeared. The appearance conditions are compiled such that on the basis of them the system links as few sensor observations as possible to a new object that has appeared, and such that this is not done other than when the probability is sufficiently great that the observations are of the type caused by a new object that has appeared. In a situation in which the linking of observations to an object that has appeared is suited to a number of appearance conditions of an object type with only a minor probability of divergence to each other, the alternative object types in question are recorded in the information of the object, and on the basis of observations later linked to the object, when this is justified according to the observations, the object types deemed to be less probable for the object are excluded.
In one preferred embodiment of the invention, the sensor observations linked to an object are used in detecting the properties of the object. The characteristics observed and recorded in the status information of an object can be the extent, shape, height, composition, distribution of mass, ability to move, distribution probability or some other characteristic of the object that is projected to the sensor field, about which there is a need to obtain information. The system processes sensor observations such that some characteristic or some characteristics of the object are determined on the basis of the correlation between the observations linked to an object known by the system and the characteristic, and on the basis of the observation series formed by the observations. The system can process sensor observations such that a correlation model used in detecting the characteristics is formed on the basis of the observation material and the basis of the known characteristics of the objects that caused the observations.
In one preferred embodiment of the invention when linking observations to the objects that are to be tracked, conditions are used that contain information about the characteristics of the space to be monitored, such as about a route leading away from the space, or about a structure or furniture, from the influence of which the object can stop causing sensor observations after moving from its previous location. An example of this type of structure is a stairway leading to the second floor and a high-legged seat is a corresponding example of the furniture.
In one preferred embodiment of the invention when linking observations to the objects that are to be tracked, conditions are used that contain information about the characteristics of the space to be monitored, such as about a route leading away from the space, or about a structure or furniture, after moving into the sphere of influence of which the object stops causing observations and after moving out of the sphere of influence of which the object causes sensor observations. Furthermore, in one embodiment the conditions described above are used as a condition of the disappearance and appearance of a new object.
In one preferred embodiment of the invention when linking observations to the objects to be tracked conditions are used that contain information about the area to be delimited to the space to be monitored, to which there is no other route probably used by the objects than the access via the monitored space. The delimited area can be a bathroom, a balcony, a cupboard or corresponding. In this embodiment information describing the movement of an object deemed to have moved into the delimited area on the basis of the tracking, which is used in linking later sensor observations to the objects, is recorded in the status information. In a case in which a sensor observation is so located on the route leading to the delimited area, on the basis of the sensor observations it is firstly diagnosed that the object that has moved to the delimited area has returned to the monitored area and secondly—if an object has not been recorded as moving to the delimited area, or the observations cannot be linked with sufficient probability to the object that moved there—it is diagnosed that a new object has appeared.
It is obvious to the person skilled in the art that the exemplary embodiments presented above are, for the sake of clarity, comparatively simple in their structure and function. Following the model presented in this patent application it is possible to construct different and also very complex solutions that utilize the inventive concept presented in this patent application.
| Number | Date | Country | Kind |
|---|---|---|---|
| 20080164 | Feb 2008 | FI | national |
| 20080236 | Mar 2008 | FI | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/FI2009/050157 | 2/25/2009 | WO | 00 | 8/27/2010 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2009/106685 | 9/3/2009 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4888581 | Guscott | Dec 1989 | A |
| 6515586 | Wymore | Feb 2003 | B1 |
| 6882959 | Rui et al. | Apr 2005 | B2 |
| 6954148 | Pulkkinen et al. | Oct 2005 | B2 |
| 7035764 | Rui et al. | Apr 2006 | B2 |
| 7394046 | Olsson et al. | Jul 2008 | B2 |
| 7617167 | Griffis et al. | Nov 2009 | B2 |
| 7684590 | Kampchen et al. | Mar 2010 | B2 |
| 7944468 | Hoffman et al. | May 2011 | B2 |
| 8077981 | Elangovan et al. | Dec 2011 | B2 |
| 8244495 | Goldbach et al. | Aug 2012 | B2 |
| 20030227386 | Pulkkinen et al. | Dec 2003 | A1 |
| 20040220769 | Rui et al. | Nov 2004 | A1 |
| 20050052533 | Ito et al. | Mar 2005 | A1 |
| 20050114079 | Rui et al. | May 2005 | A1 |
| 20060171570 | Brendley et al. | Aug 2006 | A1 |
| 20070011722 | Hoffman et al. | Jan 2007 | A1 |
| 20080128546 | Olsson et al. | Jun 2008 | A1 |
| 20080136813 | Goldbach et al. | Jun 2008 | A1 |
| 20100007476 | Klotz et al. | Jan 2010 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20110004435 A1 | Jan 2011 | US |
