Patent Grant
7123746
1. Field of the Invention
The present invention relates to a process and a system for detecting an object in relation to a surface, said surface is not necessarily flat. In accordance with the present invention an object is defined as: a material element devoid of life and/or a person or animal. The object in accordance with the invention may be static and/or in movement. The surface in accordance with the invention may be a virtual surface or a real surface transparent or not.
2. Discussion of Background
It is necessary to detect presence and determine the position of an object in relation to a surface in a number of industrial applications, notably in order to make it possible to assess whether the object is moving in relation to this surface. The study of this relative movement gives information on the object and/or the surface.
In order to present the problem posed and resolved by the present invention in the framework of a concrete industrial application, the specific case of electronic surveillance, designed to detect an intruder moving in a defined environment, will be developed hereinafter. These electronic surveillance systems make it possible to distinguish between persons behaving abnormally in relation to those behaving normally.
More specifically, the process and the system according to the invention are applicable to the surveillance of a swimming pool in order to prevent drawings.
The patent SP2 FR 96 10442 submitted on Aug. 26, 1996 on behalf of Poséidon describes this type of surveillance system for a swimming pool while referring to the previous art. Large extracts of it will be incorporated here. Surveillance of swimming pools is either non-existent, or carried out by means of human surveillance. Such surveillance is a difficult task and requires sustained attention and causing nervous fatigue in people, notably in lifeguards, who are responsible for it. In fact, in addition to the inherent limitations of any system based on human intervention, for example a reduction of vigilance due to tiredness or momentary distraction, the surveillance of swimming pools is made extremely delicate because of the reflection of light on the surface of turbulent water, a phenomenon which makes it difficult visually to locate an immobile body or a body sinking passively, at a few metres depth. The surveillance problem in swimming pools mainly arises for swimming pools that are open to the public.
In general, a swimmer in distress sinks to the bottom but, more rarely, it occurs that he floats unconsciously in a characteristic position just below the surface of the water.
An experienced lifeguard has two to three minutes to save a drowning person from the loss of consciousness, which marks the beginning of drowning. If this period is adhered to, the person drowning will not generally experience any after-effects of the accident, after a possible stay in the hospital to clean the lungs. In general, if the help is provided in three to five minutes after the loss of consciousness, however this period varies for each individual, the drowning person may still be saved, but certainly risks irreversible injuries, notably brain damage. Over five minutes, the risk of death becomes considerable.
U.S. Pat. No. 5,043,705 already proposed the use of sonar to carry out surveillance in a swimming pool. According to this system, a sonar transceiver is at least installed in the bottom of the swimming pool and a horizontal section is watched using this machine. The implementation of such a system does present inconveniences. In effect, the signal obtained with a sonar includes the echoes from the swimming pool walls and it is extremely difficult to eliminate the noise signal received in this way in order to detect the signal corresponding to the submerged body of a person drowning. In addition, the sonar essentially makes it possible to locate the body of a drowning person through the volume of air which he holds; if a person in distress has lungs filled with water, the signal obtained will not conform in any way with that which may be expected and might not even be located by processing the signal. Consequently it may be established that such a system is not likely to be satisfactory.
The patent application WO 95/34056 also proposed the implementation of cameras applicable in the field of visible waves for the surveillance of a swimming pool. These cameras are installed so that the zone observed is situated in a volume near and parallel to the bottom of the swimming pool.
In this system the cameras only observe a section of water parallel to the bottom, which implies multiplication of the cameras if the bottom is not flat and leaves the majority of the volume of the swimming pool unwatched. In addition, this system does not make it possible to detect an immobile body floating just below the surface of the water. This system consequently cannot be satisfactory.
A surveillance system for a swimming pool is described in patent application SP2 (FR 96 10442 Poséidon) which consists of:
The means of comparison of the digitised images between successive moments are appropriate only to take forms into consideration, of which the dimensions at least correspond to those of a child, in order to eliminate false alarms being caused by foreign bodies. In addition, these means of comparison are implemented in order to isolate a form and follow its trajectory at successive moments.
The means of estimating are appropriate to determine the slow nature of a movement and/or the immobility of a human body in the swimming pool by means of the results of the means of comparison.
The means for making decisions are appropriate for setting off an alarm if the slow nature of a movement or the almost-immobility of a body in the swimming pool continues over a defined period, in particular over 15 seconds.
One or numerous control screens are positioned by the lifeguards' chairs or in the offices of the persons responsible for surveillance of the swimming pool, screens on which the images are shown of a zone considered suspect. The alarm may be given through a sound and/or visual warning, in particular with an indication of the zone of the swimming pool in which a suspicious event is taking place.
No matter how perfected they are, such systems cannot always make it possible to distinguish an object in front of the bottom. In fact, in the event that a single camera is used, it is not always possible to distinguish a shadow of a body of a swimmer passively floating, moving along the bottom. Admittedly, in the event that numerous cameras are used to observe the same object from numerous points of view, the parallax effect should, in principle, make it possible to distinguish a dense object situated in front of the bottom of a bi-dimensional mark situated on the bottom. In any event numerous conditions must be met in order for the parallax effect to adequately sensitive. On the one hand, it is desirable for the viewpoints to be close to the object being observed. This first condition implies that numerous cameras will be used for surveillance over a large zone in relation to the dimensions of the objects that one hopes to detect. In correlation, this type of system is consequently particularly costly. On the other hand, it is desirable that the depiction of colours perceived by each camera is identical. Now, in order for this to be the case it is essential for the opto-electronic characteristics of the video cameras to be the same, which is not always the case. In addition the optical route between the object and each camera may cross environments with different refraction or transparency indexes. This is notably the case when the body being observed is submerged in a swimming pool with a turbulent surface. The depiction of the colours of the object being observed by each camera is not the same. Consequently the geometric correlations that make it possible to establish that the images (their outlines and grey scale nuances) produced by each camera come from the same dense object situated in front of a coloured bottom, can not longer be verified with certainty. Consequently confusion is possible between a shade of colour (for example a shadow being carried) on the bottom of the swimming pool and a dense object close to the bottom. Consequently, the result is that errors in detection and false initiation of the alarm systems.
The present invention is intended to avoid the inconveniences of the processes and systems known as yet, no matter how efficient they are.
The process and the system according to the invention were originally conceived for surveillance in swimming pools and the prevention of incidents of drowning. In any event, it quickly became apparent that the image processing techniques consisting of analysing the image of a dense object in a general way in relation to a surface, could be applied to surfaces including virtual surfaces and objects located before or behind such surfaces. Due to this fact they are appropriate for application in fields other than the surveillance of swimming pools.
Process
The present invention relates to a process for detecting an object in relation to a surface, notably a virtual surface, said surface is not necessarily flat.
The process according to the invention includes a step for realising a computer-generated image associated to the viewpoint being considered for each viewpoint by means of a geometric conversion at least including a projection in perspective of said object on said surface from at least two distinct viewpoints.
Further in the description the computer-generated image associated to the viewpoint considered is defined as the image obtained by the application of the geometric conversion at the scene being observed from said viewpoint whether or not there is an object.
In addition said process includes a stage for distinguishing between the computer-generated images for detecting a presence and determining the position of said object in relation to said surface.
In order to distinguish between the different computer-generated images said process additionally includes the following stages:
In this way each computer-generated image is broken down in virtual fields (zi) each consisting of as many specific virtual directions (di)as there are viewpoints.
The process according to the invention also includes the following stages in order to distinguish between the computer-generated images:
If opposing differences appear as a result of the search it is possible to conclude that there is an object.
Preferably, in order to determine whether an object is before and/or behind said surface, zones of comparison are selected for each of the specific virtual directions (di) on one side and/or the other side of said centres (ci) of said virtual fields (zi) in relation to each of the viewpoints.
Preferably, in order to determine whether said object extends to further or less far in relation to said surface, zones of comparison more or less distant from said centres (ci) of the virtual fields (zi) are chosen for each of the specific virtual directions (di).
Preferably, the process according to the invention also includes the stage of deducing the position of said object along the surface in relation to the position of the fields on the surface (zi) for which opposing differences have been found.
Preferably, in order to realise each of said computer-generated images associated to a viewpoint being considered,
Preferably an optical camera and/or a sonar is/are implemented at each viewpoint in order to capture the electromagnetic rays, notably lights, and/or mechanical vibrations, notably ultrasounds.
Preferably, when said computer-generated images show different coloured zones, in order to search for the opposing differences in each computer-generated image of the selected couple, inversions of colour contrasts between the zones situated along each of the specific virtual directions (di) of each field (zi) are searched for.
According to a realisation variant, the invention also relates to a process more specifically intended to detect an object in front of a bottom, in this case said bottom constitutes said surface.
In the event of this realisation variant, said geometric conversion also includes a projection, on a surface of reference (notably a flat surface) associated in the viewpoint being considered, with the virtual image resulting from the projection in perspective of said object on said bottom from said viewpoint being considered. Favourably, the projection on the surface of reference is an orthogonal projection. Equally in the event of this realisation variant, said computer-generated image associated to the viewpoint being considered consists of:
Preferably, data relating to the computer-generated image on said surface of reference is calculated on the basis of said digitised data in order to realise said computer-generated image on a surface of reference, in the event of this realisation variant.
The invention is also intended for an application implementing the aforementioned technical features and intended for the surveillance of bodies in a swimming pool. In the event of this application, said bottom consists of the bottom and/or sides of the swimming pool. In the event that, after discrimination of the computer-generated images, it is concluded that there is a body in the swimming pool:
Preferably this application relates to the event in which the body is situated close to the bottom.
Preferably in the event of this application:
The invention also relates to a system for detecting an object in relation to a surface, notably a virtual surface, said surface not necessarily being flat.
The system according to the invention consists of means of detection and the first means of calculation of a central processing unit in order to realise a computer-associated image associated to the viewpoint being considered by means of a geometric conversion consisting of at least a projection in perspective of said object on said surface from at least two distinct viewpoints.
In addition said system includes means for discriminating between the computer-generated images. The said means for discriminating make it possible to detect the presence and determine the position of said object in relation to said surface. Preferably, said means of discrimination include means of division in order to break down said surface into a mosaic of fields (Zi) each including a centre (Ci) and as many specific directions (Di) as there are viewpoints. Each computer-generated image is also broken down into virtual fields (zi) each including as many specific virtual directions (di) as there are viewpoints.
In addition said means of discrimination include discrimination between computer-generated images:
The presence of an object is the conclusion if opposing differences appear during the search.
Preferably, in order to determine whether an object is before and/or behind said surface, said means for choosing the zones on each of the specific virtual directions (di) choose the zones of comparison on one side and/or the other of said centres (ci) of said virtual fields (zi) in relation to each of the viewpoints.
Preferably, in order to determine whether said object extends further or less far in relation to said surface, said means for choosing the zones on each of the specific virtual directions (di) choose zones of comparison that are more or less distant from said centres (ci) of the virtual fields (zi).
Preferably, the means of discrimination to discriminate between the computer-generated images defining the position of said object along the surface in relation to the position on the surface of the fields (zi) for which opposing differences have been found.
Preferably, in order to realise each of said computer-generated images associated to a viewpoint being considered, said system includes sensors, at each viewpoint. The said sensors include means for capturing radioactive emissions and/or electromagnetic rays (notably lights) and/or mechanical vibrations emitted from said object, so as to produce a signal. The said system also includes:
Preferably, said sensors for capturing electromagnetic rays, notably lights, and/or mechanical vibrations, notably ultrasounds, are optical cameras and/or sonars situated at each viewpoint.
Preferably, said means of discrimination including the means for studying the inversions of the colour contrasts between the zones situated according to each of the specific virtual directions (di) of each field (zi) in order to study the opposing differences on each computer-generated image of the couple selected if said computer-generated images show different coloured zones.
According to a realisation variant the invention also relates to a system more particularly intended to detect an object in front of a bottom, in this case said bottom consists of said surface.
In the event of this realisation variant, said geometric conversion also includes a projection, on a surface of reference (notably a flat surface) associated in the viewpoint being considered, with the virtual image resulting from the projection in perspective of said object on said bottom from said viewpoint being considered. Favourably, the projection on the surface of reference is an orthogonal projection. Equally in the event of this realisation variant, said computer-generated image associated to the viewpoint being considered consists of:
Preferably, data relating to the computer-generated image on said surface of reference is calculated on the basis of said digitised data in order to realise said computer-generated image on a surface of reference, in the event of this realisation variant.
The invention is also intended for an application implementing the aforementioned technical features and intended for the surveillance of bodies in a swimming pool. In the event of this application, said bottom consists of the bottom and/or sides of the swimming pool. In the event that, after discrimination of the computer-generated images, it is concluded that there is a body in the swimming pool, said system also includes:
Preferably this application relates to the event in which the body is situated close to the bottom.
Preferably in the event of this application:
Other characteristics and advantages of the invention become apparent on reading the description of the realisation variants of the invention, given as an indicative and unrestrictive example, and from:
b which represents a perspective view of an object situated in front of a bottom after orthogonal projection on a first level of reference associated to a first viewpoint and on a second level of reference associated to a second viewpoint,
a and 5b which represent the views below the two levels of reference represented in
In the event of the realisation variation represented in referring to
An object 1 is represented in front of a bottom 2 in
The reference 3b1 (respectively 4b2) shows the section of the bottom (not masked), seen from viewpoint 3 (respect. 4), which includes a “colour mark”.
The reference 3b2 (respectively 4b2) shows the section of the bottom (not masked), seen from viewpoint 3 (respect. 4), which does not include a “colour mark”.
The virtual representation 3b, 4b of sections of the bottom not masked by said projection in perspective of object 1 is the union of zones 3b1 and 3b2 (viewpoint 3) and zones 4b1 and 4b2 (viewpoint 4).
The
In the event that the geometric conversion only consists of the projection in perspective described above, the computer-generated image, according to the present invention (classified as 3c, 4c hereinafter) consists of projections in perspective 3a and 4a of the object on said bottom.
b shows how the computer-generated image 3c, 4c is realised in the event that the geometric conversion includes an orthogonal projection 6 on the surfaces of reference 3e, 4e associated with viewpoints 3 and 4 after an operation of projection in perspective.
In the example of realisation represented in
It is also possible to use a sonar in each viewpoint 3, 4 and to capture the echo emitted by the different sections of body 1 and bottom 2 and to produce a representative signal of their form and content.
In the known way, the signal produced by the camera and/or the sonar is digitised then the digitised data obtained in this way is used to calculate said computer-generated images. If bottom 2 is flat and/or if an analytical representation of the bottom is known, it is possible directly to calculate the virtual projections 3a, 4a of the object on the bottom 2 and/or the virtual representations 3b, 4b of the sections of the bottom not masked by said virtual projections. Then, it is possible subsequently to calculate the computer-generated images 3c and 4c by means of orthogonal projection 6 of the virtual projections 3a, 4a and the virtual representations 3b, 4b on the levels of reference 3e and 4e, for which an analytical representation is also known. If the bottom is not flat as that shown in
The specific virtual directions 3di and 4di of each of the virtual fields zi of the computer-generated images associated to the virtual fields Zi of the mosaic forming the bottom are obtained by means of orthogonal projection on the levels of reference 3e and 4e of specific directions 3Di and 4Di defined above.
If there is no object 1 in front of the bottom (case as shown in
If there is an object 1 in front of the bottom (case shown in
Proceed as follows in order to distinguish between the computer-generated images 3c, 4c and in order to determine the presence of an object 1 in relation to surface 2: Select a couple of computer-generated images 3c, 4c associated to at least a couple of viewpoints 3, 4. Study (see
More particularly in the event that the computer-generated imaged 3c, 4c present zones 3fi, 3gi, 4fi,4gi of different colours, proceed as following in order to search out opposing differences in each computer-generated image 3c, 4c of the couple selected. Inversions of colour contrasts between zones 3fi, 3gi,4fi, 4gi situated along each of the specific virtual directions 3di, 4di of each field zi are sought. Computer-generated images presenting zones of different colours are notably obtained when video cameras are used.
If opposing differences between one computer-generated image and another appear for a certain number n of fields zi (n may be equal to 1), for example if 3fi is grey while 3gi is not and if 4fi is not grey while 4gi is, it must a fortiori be concluded that there is an object in front of the bottom. In analysing the colour contrasts along the specific virtual directions 3di and 4di, in seeking the inversion of contrasts it is possible to deduce whether or not there is an object 1 in front of the bottom. In any event, searching out the opposing differences may relate to other elements that the colours and shades of grey. It may also be possible to find opposing differences in the forms, in the surroundings of the outlines.
An application of the process and system according to the invention to the surveillance of bodies 1 in a swimming pool 61 was represented in referring to
In the event of the application of the process and the system according to the invention for the surveillance of a swimming pool, the two optical cameras 31 and 41 located in boxes 32 and 42 associated to viewpoints 3 and 4, are situated in the sides 62 forming a 90° angle between them. Such a situation is appropriate to sweep at least a fraction of the volume of water in the swimming pool. In this regard we refer to the patent application SP2 (FR 96 10442 Poséidon) incorporated here by reference. The levels of reference 3e and 4e are roughly parallel to the surface 63 of the water in the swimming pool.
Now a schematic view of the opto-electronic systems and the means of calculation according to a realisation variant of the invention will be described here while referring to
Cables 33, 43 which cross boxes 32, 42 containing cameras 31, 41, are coax cables, connected to a micro computer 72, for example of the “IBM compatible” type constructed around a PENTIUM microprocessor, by means of a multiplex system 71. A continuous power supply is established on each cable 33, 43 intended to supply the corresponding camera 31, 41. The said camera 31, 41 sends a modulation, which constitutes the signal to be processed through the cable 33, 43. Before input into the multiplex system 71, the separation of the continuous component must be assured thanks to the demodulation means which supply the signal emitted by the camera of the “CCD” type, solely by means of a multiplex system. The microcomputer 72 includes a central processing unit 73, means for temporary storage, or RAM (Random Access Memory), means for permanent storage, or hard disk 75, and a remote control card 76 which is able to control the alarm systems 77. In addition, the microcomputer 72 is connected to a control screen E, said screen is a touch screen which allows control of operation. The microcomputer 72 is configured as “multimedia” and is equipped with a video capture card 78, which converts these into digital images.
Thanks to the multiplex system it is possible to process cameras 31 and 41 with the same video capture card 78. It should be noted that the number of cameras processed by the same card may be greater than two.
The central processing unit 73 makes it possible to carry out various calculations making it possible to produce computer-generates images 3c, 4c. The central processing unit 73 also carries out the calculations which make it possible to distinguish between the computer-generated images 3c, 4c, according to whether or not there is an object in front of the bottom, by applying the procedures described above.
The storage means 74, 75 of the digital data of computer-generated images at successive moments t, (t+Dt) have been provided. These storage means 74, 75 consist of the microcomputer 72's memory systems, notably the internal RAM memory 74 and the hard disk 75 of the computer. The means of comparison between the computer-generated images at successive moments t and (t+Dt) have been provided. The means of comparison are formed by the central processing unit 73 of the computer and appropriate software stored in an areas of the internal RAM 74. The time interval Dt between two moments t and (t+Dt) taking into consideration is adequate in order, in the event of a normal movement of a swimmer, the differences between the two successive images translate such a movement. However, the interval of time Dt is also as small as possible in order to make it possible to set off the alarm without any delay in the event of a suspicious situation. This interval Dt may be a few tenths of a second. The means of comparison, between two moments t and (t+Dt), calculate the difference between the matrices of the two computer-generated images. The means of comparison make it possible to obtain the zones of change between two images at successive moments in this way, in other words the zones of movements between two moments being considered. The central processing unit 73 combined with appropriate software, also constitutes means of estimating the nature of a body 1, for which the computer-generated image is obtained (human body or not), of the trajectory and of the changes of attitude of this body. In addition the central unit 73 and the software are provided to constitute the means for making decisions appropriate for setting off an alarm in the event of a suspicious trajectory or movements of the body being observed. The fact that the matrix of the initial image is known (empty swimming pool), makes it possible to count and follow the different forms moving in the swimming pool captured by the cameras individually.
| Number | Date | Country | Kind |
|---|---|---|---|
| 99 16124 | Dec 1999 | FR | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/FR00/03602 | 12/20/2000 | WO | 00 | 6/17/2002 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO01/46908 | 6/28/2001 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5325449 | Burt et al. | Jun 1994 | A |
| 5369623 | Zerangue | Nov 1994 | A |
| 5754680 | Sato et al. | May 1998 | A |
| 6014472 | Minami et al. | Jan 2000 | A |
| 6133838 | Meniere | Oct 2000 | A |
| 6205242 | Onoguchi | Mar 2001 | B1 |
| 6642847 | Sison | Nov 2003 | B1 |
| 6768813 | Nakayama | Jul 2004 | B1 |
| 6793350 | Raskar et al. | Sep 2004 | B1 |
| 20040247174 | Lyons et al. | Dec 2004 | A1 |
| Number | Date | Country |
|---|---|---|
| 195 17 028 | Nov 1996 | DE |
| 0 360 880 | Apr 1990 | EP |
| 2 741 370 | May 1997 | FR |
| 11 102440 | Apr 1999 | JP |
| 11-102440 | Apr 1999 | JP |
| 11-328365 | Nov 1999 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20020191815 A1 | Dec 2002 | US |
