The present invention relates to a movable body including an imaging system.
An inspection technique using a terahertz wave is known. The terahertz wave can be defined as an electromagnetic wave having a frequency of 30 GHz (inclusive) to 30 THz (inclusive). Japanese Patent Laid-Open No. 2004-286716 discloses a method of inspecting a prohibited drug such as a narcotic drug enclosed in a sealed letter. In this method, a characteristic absorption spectrum that a prohibited drug such as a narcotic drug has in the terahertz band is used to identify a substance in a sealed letter without breaking the seal.
Recently, a dangerous item such as a knife which is taken into a movable body is a serious problem from the viewpoint of crime prevention. There is strong demand for a technique of detecting such a dangerous item in a movable body, but no such technique has been implemented.
The present invention provides a technique advantageous for crime prevention in a movable body.
A movable body according to one aspect of the present invention comprises an imaging system configured to acquire an image formed by a terahertz wave. The imaging system can be arranged to capture an inspection object inside the movable body.
One of aspects of the present invention provides a movable body comprising an imaging system configured to acquire an image formed by a terahertz wave, wherein the image is an image obtained by capturing an inspection object inside the movable body.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Hereinafter, embodiments will be described in detail with reference to the attached drawings. Note, the following embodiments are not intended to limit the scope of the claimed invention. Multiple features are described in the embodiments, but limitation is not made an invention that requires all such features, and multiple such features may be combined as appropriate. Furthermore, in the attached drawings, the same reference numerals are given to the same or similar configurations, and redundant description thereof is omitted.
In the specification, “embodiment” may be an embodiment of the invention described in the appended claims, or may be an embodiment of an invention that is not described in the appended claims.
The coach 100 can include, for example, a side door (side-sliding door) 110, a pass-through door 111, a seat 112, a facility 113, a partition door 114, a cabin aisle 115, and a deck 116. The side door 110, the pass-through door 111, and the partition door 114 are doors. In this specification, to identify the doors from each other, unique names are given according to conventions. The side door 110 is a door arranged between the inside and the outside of the coach 100. The pass-through door 111 is a door arranged at one or both of the two end portions of the coach 100. The seat 112 can be used as, for example, a non-reserved seat or a reserved seat. The facility 113 can include, for example, a wash stand, a rest room, a smoking room, and the like. The partition door 114 is a door arranged between the deck 116 and the cabin (in other words, the cabin aisle 115) in which a plurality of seats 112 are arranged. The cabin aisle 115 is an aisle provided in the cabin so as to pass by a side of the space where the plurality of seats 112 are arranged. The cabin aisle 115 can be arranged, for example, between a first seat row formed by a plurality of first seats and a second seat row formed by a plurality of second seats. The deck 116 is an aisle partitioned by, for example, the partition door 114, the pass-through door 111, and the side door 110. Note that a coach will be described here as an example. However, this technique can be applied to a movable body (automobile (standard-size vehicle, bus, and truck), airplane, helicopter, and ship) that carries persons or goods.
An imaging system ICS' can also be arranged on the platform PF. The imaging system ICS can include one or a plurality of imaging units 3a, 3b, 3c, 3d, and 3e. The imaging system ICS' can include one or a plurality of imaging units 3f The imaging units 3a, 3b, 3c, 3d, 3e, and 3f will be referred to as an imaging units 3 if they are explained without being distinguished from each other. A terahertz wave passes through a fabric, a leather, a chemical fiber, a resin, and the like. For this reason, a processor (not shown) connected to the imaging system ICS can detect a dangerous item such as a firearm, a cutting tool, or an explosive based on an image provided from the imaging system ICS.
The imaging unit 3 may be a passive type imaging unit, or may be an active type imaging unit. In the passive type imaging unit 3, an image formed on the imaging plane of the imaging unit by the terahertz wave TW radiated from the peripheral environment or the inspection object M is acquired, that is, captured as an electrical image by the imaging unit without illuminating the inspection object M with the terahertz wave TW. The active type imaging unit 3 can include an illumination source 1 and a camera 2. In the example shown in
The plurality of cameras 2 can be arranged such that their optical axes face directions different from each other. The illumination source 1 radiates the terahertz wave TW, and the inspection object M can be illuminated with the terahertz wave TW. The camera 2 acquires, that is, captures, as an electrical image, an image formed on the imaging plane by the terahertz wave TW mainly specularly reflected by the inspection object M illuminated with the terahertz wave TW. The imaging system ICS may include a visible light camera that captures an image formed by visible light. Similarly, the imaging system ICS' may include a visible light camera that captures an image formed by visible light.
The coach 100 can include a common-use portion. The common-use portion can include, for example, an aisle. The aisle can include, for example, the deck 116 and/or the cabin aisle 115. The deck 116 can include a first aisle 116-1 extending in the first direction (the horizontal direction in
The inspection object M on the platform PF can get into the coach 100 via an opening portion formed as the side door 110 opens, move through the second aisle 116-2, and change the advancing direction to a direction facing the partition door 114 at a connecting portion CP between the first aisle 116-1 and the second aisle 116-2. After that, the inspection object M can enter the cabin aisle 115 via an opening portion formed as the partition door 114 opens. Alternatively, the inspection object M on the platform PF can get into the coach 100 via the opening portion formed as the side door 110 opens, move through the second aisle 116-2, and change the course to the direction of the first aisle 116-1 (the side of the facility 113) at the connecting portion CP. That is, the inspection object M can change the advancing direction at the connecting portion CP between the first aisle 116-1 and the second aisle 116-2. The connecting portion CP can be considered as a branch point or a corner where the inspection object M changes the advancing direction. That is, the connecting portion CP can be a position where the inspection object moving in the coach (movable body) makes a direction change, a position where the inspection object decelerates or stops, or a position where the inspection object rotates. Alternatively, the connecting portion CP can be a position where the inspection object moving in the coach (movable body) is rectified. Rectify here means that spreading lines of a plurality of inspection objects lining up on the platform are reduced when they enter the coach 100. Typically, the inspection objects are rectified into one or two lines in the coach 100.
When the imaging system ICS is arranged to capture the inspection object M at the connecting portion CP, the inspection object M can be captured from various directions in accordance with a change in the direction of the inspection object M. In addition, when the plurality of cameras 2 of the imaging system ICS are arranged to capture the inspection object M at the connecting portion CP, the inspection object M can further be captured from various directions/angles. This can improve the detection probability of the position/shape/material of a dangerous item by a processor connected to the imaging system ICS.
The first aisle 116-1 and the second aisle 116-2 may intersect at the connecting portion CP, an end portion of the first aisle 116-1 may end at the connecting portion CP, and an end portion of the second aisle 116-2 may end at the connecting portion CP. Alternatively, an end portion of the first aisle 116-1 and the second aisle 116-2 may end at the connecting portion CP. Furthermore, the cabin aisle 115 may be understood as the second aisle, and the second aisle and the first aisle 116-1 may be connected at the connecting portion CP. Another example of the connecting portion CP can include a connecting portion between a planar aisle serving as the first aisle and a staircase serving as the second aisle.
In the example shown in
In some airports, a body scanner using a millimeter wave is used. Since such a body scanner is extremely bulky and inspection therewith is time consuming, it is not realistic to apply this to a ground transportation system that transports an enormous number of people. In the example shown in
The imaging unit 3e can be arranged in the cabin to capture the inspection object M via an opening portion formed as the side door 110 opens. Since the inspection object M normally often stops by instinct before the side door 110, the imaging unit 3e is advantageous in capturing a number of images. One or a plurality of illumination sources 1g and 1h configured to assist image capturing by the imaging unit 3e can be arranged on the platform PF. Also, a reflecting surface MR configured to assist image capturing by the imaging unit 3e can be arranged on the platform PF. The reflecting surface MR can include a curved surface. The reflecting surface MR may be provided on the coach 100. The reflecting surface MR can be formed by a metal surface. A film of coating or the like or a poster made of paper or the like may be provided on the metal surface. Alternatively, for example, the reflecting surface MR may be formed by a surface of a member made of a resin or the like having a surface roughness equal to or less than the wavelength of an electromagnetic wave to irradiate, preferably 1/10 or less of the wavelength, and typically on an order of 10 to 100 micrometers.
The coach 100 can include a sensor 30 configured to detect the inspection object M. The plurality of illumination sources 1a to 1d can be controlled based on the output of the sensor 30. For example, the plurality of illumination sources 1a to 1d can be controlled to radiate the terahertz wave TW in response to detection of the inspection object M by the sensor 30. The sensor 30 may also serve as a sensor configured to detect the approach of the inspection object M and open the partition door 114.
The processor 10 can specify the seat of the inspection object M based on correspondence information that associates the feature information of the inspection object M with seat information assigned to a passenger having a feature corresponding to the feature information. The feature information can be information extracted by the processor 10 from an image captured by the imaging system ICS. The feature information may be, for example, a feature amount specified based on the shape, the size, and the like of a partial image extracted from an image captured by the imaging system ICS, may be information that specifies the type of a dangerous item, or may be information representing another feature. Alternatively, the feature information may be information representing the above-described risk. Extraction of the partial image from the image captured by the imaging system ICS can include, for example, extracting a portion having a brightness more than a predetermined brightness. AI (Artificial Intelligence) can be used to extract the feature information. More specifically, AI that has undergone deep learning is installed in the processor 10, and the feature information can be extracted by the AI. For example, information representing a risk in an image captured by the camera 2 appears in a different manner depending on the position and orientation of the camera 2. Hence, deep learning can be executed based on images captured by a plurality of cameras 2.
The processor 10 can transmit the result of the above-described processing to a terminal 20 set in advance via the communication unit 15. The terminal 20 can be carried by, for example, a conductor in the coach 100. The terminal 20 may include a terminal carried by a person other than the conductor in the coach 100, a terminal provided in a security office arranged in a station or the like, and a terminal provided in an administrative body such as a police station.
The station monitoring system 120 can include an imaging system 21, a control system 22, and a ticket gate machine 23. The imaging system 21 can include a camera configured to acquire an image formed by the terahertz wave TW. The imaging system 21 may include a camera configured to acquire an image formed by an electromagnetic wave (for example, visible light) of a wavelength other than the terahertz wave. The imaging system 21 can include a camera installed to capture the inspection object passing through at least the ticket gate machine 23 and configured to acquire an image formed by the terahertz wave TW. The ticket gate machine 23 can have not only a ticket gate function but also a function of reading seat information (information for specifying a reserved seat) of a ticket (including an electronic ticket held by a portable medium such as a portable terminal) held by the inspection object that undergoes ticket gating and notifying the control system 22 of the seat information.
The imaging system 21 can capture the inspection object passing through the ticket gate machine 23 using the terahertz wave TW and transmit the captured image to the control system. The control system 22 can decide the risk of the inspection object based on the image received from the imaging system 21. In addition, the control system 22 can extract the feature information of the inspection object from the image received from the imaging system 21. The feature information can be extracted by an extraction method that is the same as or similar to the extraction method of feature information by the above-described processor 10. The control system 22 can be formed by, for example, a general-purpose or dedicated computer in which a program is installed.
The control system 22 generates correspondence information that associates the feature information of the inspection object extracted from the image received from the imaging system 21 with the seat information read by the ticket gate machine 23. For example, the feature information can be information strongly suggesting holding of a firearm, and the seat information can be seat information read by the ticket gate machine 23 from a ticket held by the inspection object that holds the firearm. The correspondence information can be transmitted from the control system 22 to the coach 100. The feature information may include information that identifies the ID of the inspection object (that is, information that specifies an individual). If the imaging system 21 includes a visible light camera, the ID of the inspection object can be identified from the visible light image of the inspection object or from the visible light image by AI or the like. The visible light image of the inspection object having a predetermined risk can be transmitted to the coach 100 together with the above-described correspondence information and can further be transmitted to the terminal 20.
Hereinafter, a technique advantageous in improving crime prevention by a camera system installed in a facility will be described.
A camera system 200 according to some embodiments of the present invention will be described with reference to
The camera system 200 includes an imaging system 201 configured to acquire an image formed by a terahertz wave reflected by an inspection object 250. The imaging system 201 can include at least one illumination unit 202 configured to irradiate a terahertz wave, and at least one camera 203 configured to acquire an image formed by the terahertz wave. The illumination unit 202 is also referred to as the irradiation unit. To discriminate a plurality of illumination units 202 and a plurality of cameras 203 in the following explanation, a suffix is added to each reference numeral, like an illumination unit 202″a″ and a camera 203″a″. If the illumination units and the cameras need not be discriminated, they are expressed simply as “illumination unit 202” and “camera 203”. This also applies to other constituent elements.
In this embodiment, the camera 203 that detects a terahertz wave is of a type called active camera, and can be used in combination with the illumination unit 202. However, the camera is not limited to this, and may be a camera of a passive type. In this case, without illuminating the inspection object 250 with a terahertz wave irradiated from the illumination unit 202, and an image can be acquired by a terahertz wave radiated from the inspection object 250.
The imaging system 201 can be arranged to capture the inspection object 250 that uses the station. The inspection object 250 is normally a person but may be an animal other than a person or a robot. A terahertz wave passes through a fabric, a leather, and the like. For this reason, a processor (for example, a control system 22) (not shown) connected to the camera system 200 can detect a dangerous item such as a firearm, a cutting tool, or an explosive based on an image provided from the imaging system 201 of the camera system 200.
In the arrangement shown in
As shown in
The arrangement of the illumination unit 202 and the camera 203 is not limited to the above-described arrangement. For example, the illumination unit 202 and the camera 203 may be arranged on the ticket gate machine 211a. Alternatively, for example, the illumination unit 202 and the camera 203 may be arranged near the center of the ticket gate machine 211, or may be arranged near the end portion on the outer side of the ticket gate in
If the imaging system 201 of the camera system 200 is used as a surveillance camera, in some cases, post-processing such as image processing by a processor (not shown) at the subsequent stage of the imaging system 201 of the camera system 200 is facilitated by capturing the person who is the inspection object 250 one by one. The ticket gate machine 211 passes the person who is the inspection object 250 one by one at a high possibility. Hence, when the imaging system 201 is arranged in the ticket gate machine 211, the load of post-processing such as image processing can be suppressed. That is, the imaging system 201 can be arranged in a place where the inspection object 250 lines up. In addition, the time needed for the person that is the inspection object 250 to pass through the ticket gate machine 211 is about 1 sec. However, the imaging system 201 can acquire an image formed by a terahertz wave at a frame rate of 50 fps or more. For this reason, it is possible to perform capture one inspection object 250 at plurality of times. In the plurality of times of image capturing, the inspection object 250 may be captured wholly or may be captured only partially.
The imaging system 201 of the camera system 200 may include a sensor 260 configured to detect that the inspection object 250 approaches. For example, the ticket gate machine 211 may be provided with the sensor 260, as shown in
Additionally, in
An example in which the imaging system 201 is applied to a partition wall 213 on a platform 216 of a station will be described next with reference to
The imaging system 201 includes the illumination unit 202a and the camera 203a, which are arranged in the track-side area 217. The illumination unit 202a and the camera 203a perform illumination and image capturing of the passage 241 from the track-side area 217 when the door portion 214 opens. In addition, the imaging system 201 includes the illumination unit 202b and the camera 203b, which are arranged in the platform 216. The illumination unit 202b and the camera 203b perform illumination and image capturing of the passage 241 from the platform 216 when the door portion 214 opens. When the illumination unit 202a and the camera 203a, and the illumination unit 202b and the camera 203b are arranged, it is possible to acquire the front- and rear-side images of both the inspection object 250 that gets in a railroad coach 218 via a door 219 and the inspection object 250 that gets off the railroad coach 218 via the door 219. However, the present invention is not limited to this, and only the illumination unit 202a and the camera 203a or only the illumination unit 202b and the camera 203b may be arranged.
Each of the illumination units 202a and 202b may include a plurality of illumination devices, as shown in
In addition, as shown in
The arrangement of the illumination units 202a and 202b and the cameras 203a and 203b is not limited to the arrangement shown in
As described above, if the imaging system 201 is used as a surveillance camera, the load of post-processing such as image processing can be reduced by capturing the person who is the inspection object 250 one by one. Hence, the imaging system 201 included in the camera system 200 may be applied to the partition wall 213 installed in a station where a bullet train or a limited express for which persons line up and get in one by one stops. In this case, the width of the door 219 of the railroad coach 218 used for the bullet train or limited express is about 700 mm to 1,000 mm. Hence, in the arrangement shown in
For example, the imaging system 201 may be applied to, for example, the partition wall 213 in a railroad station of a commuter train or the like. In this case, it may be possible to acquire images of persons one by one except during rush hours. Even in a case in which a plurality of inspection objects 250 simultaneously get in or get off, the inspection objects 250 often line up in two or three lines and get in. Each inspection object 250 can be distinguished by image processing or the like using a processor included in the camera system 200. In a commuter train or the like, the width of a door is about 1,300 mm to 2,000 mm. For this reason, in the arrangement shown in
In some cases, the platform 216 is arranged outdoors. Hence, the imaging system 201 arranged on the platform 216 or the track-side area 217 is readily affected by the external environment. A terahertz wave is readily absorbed by water, and it may be impossible to obtain images with sufficient image quality in a highly humid environment such as a rainfall. Hence, the imaging system 201 may include a sensor 261 configured to detect the external environment, as shown in
In addition, for example, the illumination unit 202 or the camera 203 may be attached to the vehicle body of the railroad coach 218. That is, the camera system 200 may include an illumination unit or a camera mounted on the railroad coach 218. In this case, the camera system 200 can include a communication unit between the imaging system 201 arranged in a station and the imaging system including the illumination unit or the camera included in the railroad coach 218.
In addition, for example, the illumination unit 202 and the camera 203 may start operations when the door portion 214 of the partition wall 213 opens. For example, the imaging system 201 may synchronize with the operation of the door portion 214, or may include a sensor configured to detect that the door portion 214 has opened. This can suppress power consumption of the imaging system 201.
An example in which the imaging system 201 is applied to an escalator 221 will be described next with reference to
The imaging system 201 is arranged to be adjacent to the escalator 221 to acquire the image of the inspection object 250 that passes through the escalator 221. In the arrangement shown in
For example, as shown in
As described above, if the imaging system 201 is used as a surveillance camera, it may be advantageous that the person who is the inspection object 250 can be captured one by one. Since the escalator 221 operates at a predetermined speed, the possibility that the image of the inspection object 250 can be acquired one by one is high. Additionally, as shown in
In addition, as described above, a terahertz wave can pass through a resin or the like. For this reason, the illumination unit 202 or the camera 203 may be embedded in the floor, wall, or ceiling of the portion where the escalator 221 is arranged. For example, the illumination unit 202 may be installed on the deck board of the escalator 221 together with a normal illumination.
An example in which the imaging system 201 is applied to a staircase 222 will be described next with reference to
The imaging system 201 is arranged in the staircase 222 to acquire the image of the inspection object 250 that passes through the staircase 222. In the arrangement shown in
Since the person who is the inspection object 250 goes up or down the staircase 222 one by one (or by about two steps), the image of the inspection object 250 can sequentially be acquired from the head (or foot) of the inspection object 250 to the foot (or head).
For the window 224 provided in the riser portion 223 of the staircase 222, various kinds of resins that pass a terahertz wave can be used, as described above. When an appropriate resin material is selected in accordance with the material used for the riser portion 223 or a tread portion 225 of the staircase 222, where the imaging system 201 is not arranged, the imaging system 201 can be made unnoticeable (its existence can be hidden).
An example in which the imaging system 201 is applied to a passage 242 will be described next with reference to
The imaging system 201 is arranged in the passage 242 to acquire the image of the inspection object 250 that passes through the passage 242. The imaging system 201 includes the illumination unit 202 and the camera 203. At this time, one of the illumination unit 202 and the camera 203 is arranged on a ceiling 227 of the passage 242, and the other of the illumination unit 202 and the camera 203 is embedded in a floor 226 of the passage 242. In the arrangement shown in
In the arrangement shown in
Additionally, in the arrangement shown in
However, the arrangement of the illumination unit 202 and the camera 203 on the passage 242 is not limited to the arrangement shown in
As described above, when the imaging system 201 is arranged in the staircase 222 or the passage 242, a plurality of cameras 203 may be arranged in the widthwise direction of the staircase 222 or the passage 242. Accordingly, the possibility that the image of the inspection object 250 can be captured one by one becomes high. In addition, the imaging system 201 may be arranged in a portion of the staircase 222 or the passage 242, where the width decreases. In the portion of the staircase 222 or the passage 242, where the width decreases, the inspection object 250 can easily line up.
Additionally, the camera system 200 according to this embodiment can monitor the inspection object 250 shown in
The control system 310 can specify the inspection object 250 based on the feature information of the inspection object 250 and correspondence information that associates the feature information with seat information assigned to a passenger having a feature corresponding to the feature information. The feature information can be information extracted by the control system 310 from an image obtained by the imaging system 201. The feature information may be, for example, a feature amount specified based on the shape, the size, and the like of a partial image extracted from an image obtained by the imaging system 201, may be information that specifies the type of a dangerous item, or may be information representing another feature. Alternatively, the feature information may be information representing the above-described risk. Extraction of the partial image from the image acquired by the imaging system 201 can include, for example, extracting a portion having a brightness more than a predetermined brightness. AI (Artificial Intelligence) can be used to extract the feature information. More specifically, AI that has undergone deep learning is installed in the control system 310, and the feature information can be extracted by the AI. For example, information representing a risk in an image captured by the camera 203 appears in a different manner depending on the position and orientation of the camera 203. Hence, deep learning can be executed based on images captured by a plurality of cameras 203.
The control system 310 can transmit the result of the above-described processing to a terminal 320 set in advance via the communication unit 315. The terminal 320 can be carried by, for example, a conductor in the railroad coach 218. The terminal 320 may include a terminal carried by a person other than the conductor in the railroad coach 218, a terminal provided in a security office arranged in a station or the like, and a terminal provided in an administrative body such as a police station.
The imaging system 201 can acquire the image of the inspection object 250 that passes through the ticket gate machine 211 and transmit the obtained image to the control system 310. The control system 310 can decide the risk of the inspection object based on the image received from the imaging system 201. In addition, the control system 310 can extract the feature information of the inspection object from the image received from the imaging system 201.
The control system 310 generates correspondence information that associates the feature information of the inspection object 250 extracted from the image received from the imaging system 201 with seat information read by the ticket gate machine 211. For example, the feature information can be information strongly suggesting holding of a gun, and the seat information can be seat information read by the ticket gate machine 211 from a ticket held by the inspection object 250 that holds the gun. The correspondence information can be transmitted from the control system 310 to the terminal 320 in the railroad coach 218. The feature information may include information that identifies the ID of the inspection object 250 (that is, information that specifies an individual). The imaging system 201 may include a visible light camera, and the ID of the inspection object 250 can be identified from the visible light image of the inspection object 250 or from the visible light image by AI or the like. The visible light image of the inspection object having a predetermined risk can be transmitted to the railroad coach 218 together with the above-described correspondence information and can further be transmitted to the terminal 320. A case in which the image of the inspection object 250 is acquired by the imaging system 201 arranged on the ticket gate machine 211 has been described here. However, tracking of the inspection object 250 may be started or continued based on an image obtained from the imaging system 201 arranged on the partition wall 213, the escalator 221, the staircase 222, or the passage 242. In addition, tracking of the inspection object 250 may be started based on an image obtained by the imaging system 201 arranged on the ticket gate machine 211, and after that, tracking of the inspection object 250 may be continued using a surveillance camera using visible light.
Hereinafter, a processing system capable of more advantageously executing an inspection using a terahertz wave will be described. In the following descriptions, terahertz waves include electromagnetic waves within the frequency range of 30 GHz to 30 THz. The concept of electromagnetic waves can include visible light, infrared light, and a radio wave such as a millimeter wave.
The outline of a processing system 401 according to the first embodiment will be described with reference to
The first camera 402 of the first imaging system acquires a first image based on a terahertz wave 403 of a first wavelength radiated from the first illumination source 404. An inspection object 410 is irradiated with the terahertz wave 403 radiated from the first illumination source 404. If the inspection object 410 is a dressed person, the terahertz wave 403 passes through the fibers of clothes and is reflected by a metal or ceramic held by the inspection object 410. A specific substance, for example, RDX (trimethylenetrinitroamine) that is an explosive is known to absorb a terahertz wave near 0.8 THz, and therefore, the reflected wave decreases. The first camera 402 acquires the first image based on the reflected wave.
The second camera 405 of the second imaging system acquires a second image from an electromagnetic wave of a wavelength different from that of the terahertz wave irradiated from the first illumination source 404. As the electromagnetic wave of a different wavelength, visible light, infrared light, or a millimeter wave can be used. When using infrared light, an illumination source (not shown) different from the first illumination source 404 may be prepared. The second image acquired by the second camera 405 is processed by the pre-processing unit 406. The pre-processing unit 406 performs processing of detecting an inspection region from the second image.
If the second image is acquired by visible light, and the inspection object 410 is a person, detection of the inspection region may be performed by detecting a specific part of clothes as the inspection region. The inspection region may be specified by creating a model by machine learning and classifying a region of the captured second image by the model. Alternatively, the region may be specified based on the information of the shape of an object stored in a database 409. If the second image is acquired by a millimeter wave, a portion where the intensity distribution in the image is higher than a threshold or a portion where the intensity difference is large may be detected as the inspection region. If infrared light is used to acquire the second image, a portion with little radiation of infrared light caused by water or a specific portion of clothes in an image detected by night vision may be detected as the inspection region. Even in a dark place or a place with a poor view due to the weather, the inspection region can be detected using the infrared light or a millimeter wave. When detecting the inspection region from an image of a dressed person, an unnaturally swelling portion of clothes, the chest portion of the person, or a pocket portion of clothes may be detected as the inspection region.
The inspection of the inspection object 410 by the processor will be described based on
When the region corresponding to the inspection region is selected from the first image, and image processing is performed, the processing can be performed while reducing unnecessary information. For this reason, the processing load can be reduced as compared to processing of the entire image data, and the speed can be increased. Hence, even if the inspection object 410 is moving, features can be detected from the first image a plurality of times in a short moving distance during a short time. A determination unit 408 estimates the object under the clothes based on the plurality of detected features (step S426). The plurality of features may be features of a part of the object. The determination unit 408 may classify the shape of the object detected from the first image based on the data in the database 409. The classification may be done using a model created by machine learning. It is considered that the information of the shape obtained from the image may be the information of a part of the object because of the movement of the inspection object 410 or the positional relationship between the inspection object and the camera. Even in this case, the estimation accuracy can be improved by classifying the features based on the information of the plurality of features, accumulating a plurality of results, and performing determination based on the accumulated classification results (step S427).
When the processing system is used in a security monitoring system, the risk of the object detected from the inspection region is determined based on the accumulation of the classification results for the inspection object 410 (step S428). As for the determination, determination based on the accumulation result of classifications may be performed based on a model by machine learning. If it is determined that the inspection object 410 holds a dangerous substance, it is possible to notify the outside that the inspection object 410 holds a dangerous substance. When the inspection object 410 passes through a gate in which the processing system is arranged, the processing system may notify the outside of a warning. When the inspection object 410 puts in a ticket and passes through a ticket gate, the processing system may link the ticket with the inspection object 410 and notify that the inspection object 410 is a monitoring target. If the second image is obtained using visible light, the inspection object 410 can be displayed such that it can easily be seen by displaying the second image and the first image on a monitor in a superimposed manner. When the determination is suspended, the inspection is repeated until the end condition is satisfied. The end condition may be the number of repetitions of the inspection (S429).
In this embodiment, a second imaging system is provided with a second illumination source 411 that radiates a terahertz wave. This embodiment will be described with reference to
Processing according to this embodiment will be described based on
Data of a portion where reflection and absorption in the second image are almost equal to those in the first image is almost canceled by calculating the difference between the two pieces of information. However, data of a portion where reflection and absorption are different between the first wavelength and the second wavelength is not canceled even by calculating the difference between the two images. In this way, the spectrum analysis of the substance in the inspection region can be performed using the difference in the rate of terahertz wave absorption by the substance. The type of the substance can be estimated using the spectrum analysis. In addition, since scattering or reflection by clothes is canceled, an unnecessary signal from the clothes can be reduced from the obtained image information, and the signal-to-noise ratio of the image can be improved.
If person as the inspection object holds a substance that readily absorbs the first wavelength, the substance detected in the inspection region can be classified based on the difference in the absorption rate between the first wavelength and the second wavelength (step S436). As for the classification, when the relationship between a specific substance and a wavelength is held in a database 409, a determination unit 408 can perform the classification based on the database 409. The determination unit 408 may perform the classification using a model created by machine learning. With the above-described method, it is possible to estimate that the inspection object 410 holds the substance that absorbs the specific wavelength. It is known that dangerous substances exist among the substances that absorb a terahertz wave of a specific wavelength. The existence of a dangerous substance can be estimated by spectrum analysis. The detection accuracy can be raised by accumulating a plurality of spectrum analysis results (step S437).
It is thus determined that the inspection object 410 may hold a dangerous substance (step S438). As for the determination, determination based on the accumulation result of classifications may be performed based on a model by machine learning. If it is determined that a dangerous substance is held, the processing system notifies the outside that the inspection object 410 holds a dangerous substance. When the inspection object 410 passes through a gate in which the processing system is arranged, the processing system may notify the outside of a warning. When the person of the inspection object 410 puts in a ticket and passes through a ticket gate, the processing system may link the ticket with the inspection object 410 and notify the outside of the person as a monitoring target. As for the wavelength of the terahertz wave irradiated from the second illumination source 411, a plurality of illumination sources capable of irradiating terahertz waves of a plurality of, that is, three or more wavelengths may be combined in accordance with the absorption spectrum of a substance to be detected. When the determination is suspended, the inspection is repeated until the end condition is satisfied. The end condition may be the number of repetitions of the inspection (S439).
In this embodiment, based on detection of a specific region in a second image captured by a second imaging system a control unit 412 is controlled to control a first illumination source 404 and a first camera 402 in a first imaging system. This embodiment will be described with reference to
A second camera 405 of the second imaging system acquires a second image from an electromagnetic wave of a wavelength different from a terahertz wave radiated from the first illumination source 404. As the electromagnetic wave of a different wavelength, visible light, infrared light, or a millimeter wave can be used. The second image acquired by the second camera 405 is processed by a pre-processing unit 406. The pre-processing unit 406 detects an inspection region from the second image (steps S452 and S453). Detection of the inspection region is performed as described in the first embodiment.
Conditions at the time of capturing by the first camera are controlled in accordance with the position and range of the inspection region detected from the second image and the state of the inspection region. The conditions include control of the posture of the first camera, control of a gain for an acquired image, and control of a capturing range for zooming or trimming and an angle of view (step S454). The output level (output power) and the wavelength of the terahertz wave irradiated from the first illumination source 404 may be changed in accordance with the strength of a reflected signal from the inspection region or a target object in the inspection region. By this control, the inspection accuracy can be raised. The first imaging system controlled by the control unit 412 acquires a first image based on the terahertz wave of a first wavelength (step S455).
A post-processing unit 407 performs processing of the inspection region based on the acquired first image (step S456). After that, a determination unit 408 performs determination and classification of an object (steps S457, S458, and S459). When the processing system is a security monitoring system, a risk is determined based on the accumulation of classification results. If it is determined that an inspection object 410 holds a dangerous substance, the processing system notifies the outside that the inspection object 410 holds a dangerous substance. When the inspection object 410 passes through a gate in which the processing system is arranged, the processing system may notify the outside of a warning. When the inspection object 410 puts in a ticket and passes through a ticket gate, the processing system may link the ticket with the inspection object 410 and set the inspection object 410 to a monitoring target. When the determination is suspended, the inspection is repeated until the end condition is satisfied. The end condition may be the number of repetitions of the inspection (S460).
In this embodiment, an environment monitoring unit 413 configured to monitor a humidity around a processing unit is provided. This embodiment will be described with reference to
More specifically, if the environment monitoring unit 413 detects that the humidity has become high, the wavelength of a terahertz wave 403 radiated from a first illumination source 404 is switched to a wavelength longer than the wavelength currently under use. In accordance with the humidity, the wavelength may be switched to a wavelength (a region that exists near a wavelength of 1.2 mm or 0.75 mm, where attenuation of atmosphere is specifically small) hardly affected by water vapor. When the wavelength of the terahertz wave becomes long, the resolution of an image captured by the camera lowers. However, it is possible to reduce the influence of water vapor and continue inspection.
In this embodiment, capturing is performed using terahertz waves of different wavelengths. A second image is acquired using a terahertz wave of a second wavelength longer than the wavelength in capturing a first image, and an inspection region is detected from the second image. The inspection region may be detected as a region including an object of a predetermined shape using a model created by machine learning, or a region where the spectrum of a reflected wave of a predetermined wavelength changes may be detected as the inspection region.
This embodiment will be described with reference to
More specifically, depending on the posture of the inspection object 410, a partial shape is acquired as the shape of the object held by the inspection object 410. On the other hand, in the image obtained by capturing 2, since the wavelength of the terahertz wave is long, the resolution is low, and the shape of each object is not clear as compared to capturing 1. However, since the terahertz wave of a long wavelength is used, the depth of field is deep, and the capturing is insensitive to a change in the posture of the inspection object 410. More specifically, independently of the posture of the inspection object 410, the whole shape of the object held by the inspection object 410 is acquired. When capturing 2 of a low resolution is processed to specify the position of an object held by the inspection object 410, and the data of capturing 1 is processed based on the detected inspection region, the processing load can be reduced, and the processing can be performed at a higher speed. Hence, even if the inspection object 410 is moving, features of the inspection object 410 can be detected a plurality of times in a short moving distance during a short time, and the object under clothes can be estimated based on the detected features.
In addition, when the difference between capturing 1 and capturing 2 performed using terahertz waves of the two different wavelengths is calculated, reflection by clothes is canceled, and noise can be reduced from the obtained image information. More specifically, since scattering is the main component of reflection from whole clothes, the intensity difference is small, and the capturing is insensitive to a change in the posture of the inspection object 410 (random noise is added to the acquired image as a whole). For this reason, when the differential image between capturing 1 and capturing 2 is calculated, the signal of clothes is canceled. In addition, when the difference is calculated, an image based on the difference in the terahertz wave absorption rate of the substance through which the terahertz wave passes can be obtained. Hence, the shape of an object containing a substance other than a metal or ceramic as a component can be detected from the difference between the first image and the second image.
The object in the inspection region is estimated by a determination unit 408 by classifying the shape of the object detected from capturing 1. If the inspection object 410 moves, the shape of the object obtained from the image is often partial. Hence, the determination accuracy can be improved by accumulating a plurality of classification results and performing determination based on the accumulated classification results. In a case of a security monitoring system, a risk is determined based on the accumulation of classification results. If it is determined that the inspection object 410 holds a dangerous substance, the processing system notifies that the inspection object 410 holds a dangerous substance. When the inspection object 410 passes through a gate in which the processing system is arranged, the processing system may notify the outside of a warning. When the inspection object 410 puts in a ticket and passes through a ticket gate, the processing system may link the ticket with the inspection object 410 and set the inspection object 410 to a monitoring target.
An application example of the processing system will be described with reference to
An inspection object 410 can be tracked by the second camera 405-1, and the posture and the angle of view of the first camera 402 can be controlled. When the wavelength of the terahertz wave used for capturing by the second camera 405-2 configured to perform capturing based on a terahertz wave is set in accordance with the absorption rate of a substance, spectrum analysis can be performed. In addition, when the second cameras 405-1 and 405-2 are used to detect an inspection region, the processing load for a first image captured by the first camera 402 can be reduced.
Furthermore, the shape of an object containing a substance other than a metal or ceramic as a component can be detected using the difference in the absorption rate of the substance for the wavelength of the terahertz wave. In this embodiment, as the second cameras 405, a camera for visible light, infrared light, or a millimeter wave and a camera for a terahertz wave of a second wavelength are used. However, only one of the camera for visible light, infrared light, or a millimeter wave and the camera for a terahertz wave of a second wavelength may be used as the second camera. The illumination sources and the cameras can unnoticeably be buried in a wall surface, a ceiling, or a floor surface. The illumination sources and the cameras may be arranged on both of the left and right sides of the doorway 414. When the illumination sources and the cameras are provided near the doorway 414, situations in which a plurality of inspection objects 410 overlap can be reduced, and the inspection accuracy can be improved.
An example in which the processing system is arranged near a ticket gate machine 415 installed at a ticket gate of a station will be described with reference to
The operation of the processing system may be started in accordance with detection of a motion of the inspection object 410 by a sensor provided separately from the processing system opening/closing of a door of a vehicle, putting of a ticket into the ticket gate machine 415, or the like. A plurality of first cameras and second cameras may be provided. By using a plurality of cameras, the detection accuracy can be improved, the number of inspection objects can be increased, and the inspection region can be expanded.
The operation of the camera system 200 will be described next with reference to
First, in step S1001, the illumination unit 202 irradiates the inspection object 250 with a terahertz wave under a desired condition. Next, in step S1002, the camera 203 detects the terahertz wave reflected by the inspection object 250 and acquires information based on the terahertz wave. In step S1003, a control unit performs processing of converting the information based on the terahertz wave into an image. Here, the control unit can be, for example, the control system 310 as mentioned above.
Next, the control unit evaluates the quality of the acquired terahertz image (step S1004). As the evaluation items, items representing whether an appropriate terahertz image according to the inspection object 250 can be acquired, whether an article can be detected from the terahertz image of the image quality, and the like can appropriately be set. In the image quality evaluation, if desired image quality is not satisfied, the camera system 200 performs the operation of step S1005. In step S1005, the control unit supplies, to the illumination unit 202, a control signal for changing the wavelength and increasing the power of the irradiated terahertz wave. The illumination unit 202 performs terahertz wave irradiation (step S1001) again. With the series of operations, a desired article can appropriately be detected.
Note that in the image evaluation, upon determining that the desired image quality can be obtained, the control unit judges the presence and absence of a detected article, and in some cases, the type of the article (step S1006). If an article is detected, the control unit causes a monitor system to display an alert. Alternatively, the control unit outputs an instruction to perform an operation of adding a flag to an article or person of high risk (step S1007). If no article is detected, the control unit may add a flag to the confirmed person of low risk (step S1008).
For the series of operations, the illumination unit 202 and the camera 203 can be used in the following combinations. If there are the illumination unit 202 and the camera 203 used in the first capturing, second and subsequent capturing may be executed using the same illumination unit 202 and the same camera 203. In addition, if there are the illumination unit 202 and the camera 203 used in the first capturing, second and subsequent capturing may be executed using the same illumination unit 202 as in the first capturing and the camera 203 different from that in the first capturing. Furthermore, if there are the illumination unit 202 and the camera 203 used in the first capturing, second and subsequent capturing may be executed using the illumination unit 202 different from that in the first capturing and the same camera as in the first capturing. Furthermore, if there are the illumination unit 202 and the camera 203 used in the first capturing, second and subsequent capturing may be executed using the illumination unit 202 and the camera 203 which are different from those in the first capturing.
Another operation of the camera system 200 will be described next with reference to
First, in the camera system 200, the illumination unit 202 is in a standby state (step S1101). At this time, the camera 203 may also be in the standby state. The control unit detects a door opening signal (step S1102). Upon detecting the door opening signal, the control unit supplies a control signal for irradiating the inspection object 250 to the illumination unit 202, and supplies a control signal for capturing the inspection object 250 to the camera 203. The illumination unit 202 starts terahertz wave irradiation in accordance with the control signal from the control unit (step S1103). The camera 203 starts detecting the terahertz wave in accordance with the start of illumination by the illumination unit 202 (step S1104). If the control unit does not detect the door opening signal, the standby state is maintained (step S1101). Upon detecting the door opening signal in step S1102, the control unit is set in a state in which it can always detect the door closing signal (step S1106). Upon detecting a door closing signal in step S1106, the control unit supplies a control signal for stopping terahertz wave irradiation to the illumination unit 202, and supplies a control signal for stopping terahertz wave detection to the camera 203 (steps S1107 and S1108). Here, if the control unit detects the door closing signal, at least one of steps S1107 and S1108 is performed. If the door closing signal is not detected, irradiation and detection of the terahertz wave are continued, and the camera system 200 continues capturing. By the series of operations of monitoring the open and close state of the door, power of the camera system 200 can be saved. In addition, by the series of operations, reliable capturing can be performed at a necessary timing.
Still another operation of the camera system 200 will be described next with reference to
First, in the camera system 200, the illumination unit 202 is in a standby state (step S1201). At this time, the camera may also be in the standby state. The control unit is in a state in which it can detect a door opening signal that notifies that the door provided in the ticket gate machine 211 opens (step S1202). If the control unit detects the door opening signal, the illumination unit 202 starts terahertz wave irradiation (step S1203). In addition, the camera 203 starts detecting the terahertz wave in accordance with the start of illumination by the illumination unit 202 (step S1204). If the control unit does not detect the door opening signal, the standby state is maintained (step S1201). To detect the door opening signal in step S1202, a signal generated by a ticket put into the ticket gate machine 211, a signal generated by bringing a ticket such as an IC card into contact with the ticket gate machine 211, or a signal for detecting the presence and absence of a ticket such as an IC card using a millimeter wave in the ticket gate machine 211 can be used. When the open and close state of the door of the ticket gate machine 211 is monitored in this way, power can be saved. In addition, by the operation, reliable capturing can be performed.
Still another operation of the camera system 200 will be described next with reference to
As described above, the sensor 260 detects the inspection object 250. Here, the sensor 260 may be, for example, a motion sensor using infrared rays or a camera using visible light. First, in the camera system 200, the illumination unit 202 is in a standby state (step S1301). At this time, the camera may also be in the standby state. Next, the inspection object 250 is detected using the sensor 260 (step S1302). The signal from the sensor 260 is sent to the control unit. Upon determining that the inspection object 250 is detected, the control unit supplies a control signal for irradiating the inspection object 250 to the illumination unit 202. The illumination unit 202 starts terahertz wave irradiation in accordance with the control signal from the control unit (step S1303). In addition, the control unit supplies a control signal for capturing the inspection object 250 to the camera 203. The camera 203 starts detecting the terahertz wave in accordance with the control signal from the control unit (step S1304). If the control unit does not determine that the inspection object 250 is detected, the standby state is maintained (step S1301).
By this operation, power can be saved. Additionally, with this operation, reliably capturing can be performed. In this embodiment, a case in which the inspection object 250 is a person has been described. However, the inspection object 250 may be an object. This arrangement can also be applied to the escalator 221 shown in
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2019-047787, filed Mar. 14, 2019 and Japanese Patent Application No. 2020-032189, filed Feb. 27, 2020, which are hereby incorporated by reference herein in their entirety.
Number | Date | Country | Kind |
---|---|---|---|
2019-047787 | Mar 2019 | JP | national |
2020-032189 | Feb 2020 | JP | national |
Number | Date | Country | |
---|---|---|---|
Parent | 16815171 | Mar 2020 | US |
Child | 18307268 | US |