The present disclosure relates to an information processing apparatus, an information processing method, an imaging apparatus, a network camera system, and a storage medium.
In general, a human-body detection process of detecting a human body in an image captured by a surveillance camera by analyzing the image has been used. The human-body detection process is used in a passing detection process of counting a number of human bodies which have passed a detection line set in the image.
The human-body detection process employs a method for extracting a feature value of a human body in an image and performing template matching by comparing the feature value with a dictionary (templates) generated in advance. However, in the template matching, the comparison between a feature value extracted from the image to be processed and a dictionary is repeatedly performed, and therefore, a long period of time is required for processing. Therefore, if a large number of people are included in the image or if people and a complicated background are difficult to be distinguished from each other, the human-body detection process is not completed within a predetermined period of time. In this case, the process is interrupted and a next image is subjected to the human-body detection process, and therefore, accuracy of the passing detection process is degraded.
To address this problem, a method for improving a frame rate of a detection process by executing the human-body detection process only on a predetermined region in an image is used.
Japanese Patent Laid-Open No. 2007-280088 discloses a method for generating a range image including distance information held by individual pixels using a stereo camera and setting a size of a detection region in accordance with a size of a specific object to be detected using the distance information.
However, in a case where human bodies are detected in a predetermined region in an image and a number of human bodies which have passed a detection line in the detected human bodies are counted, a user is required to manually set a detection region to be subjected to the human-body detection process and a detection line to be used in the passing detection process and the settings are troublesome. Furthermore, it is difficult for the user to appropriately set the detection region relative to the detection line. Even by the technique disclosed in Japanese Patent Laid-Open No. 2007-280088, an appropriate detection region may not be automatically set relative to the detection line.
According to an aspect of the present disclosure, an information processing apparatus includes a detection line setting unit configured to set a detection line to be used for detecting passing of a subject in an image, a subject detection unit configured to perform a detection process of detecting the subject, a determination unit configured to determine whether the subject which has been detected in the image has passed the detection line, and a detection region setting unit configured to set, in accordance with the detection line, a detection region so as to include the detection line. The detection region is to be subjected to the detection process by the subject detection unit.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.
The network camera system is used to perform analysis of customer behavior in a retailing field, for example. Specifically, the network camera system may obtain data on a level of congestion of customers in front of a cash register in a shop for each day of the week and each time zone by counting a number of people in a region in front of the cash register. Furthermore, the network camera system may obtain the number of customers who have visited a shop by counting a number of people who have passed a predetermined position in the shop and obtain paths of movement of the customers in the shop.
The network 300 includes a plurality of routers, a plurality of switches, a plurality of cables, and the like which are based on a communication standard of Ethernet (registered trademark). Note that the network 300 may be based on any communication standard and have any scale and any configuration as long as the camera 100 and the client apparatus 200 may communicate with each other. The network 300 may be realized by the Internet, a wired local area network (LAN), a wireless LAN, a wide area network (WAN), or a combination of them.
The camera 100 is an imaging apparatus which captures an image in a predetermined imaging range. The camera 100 may be a pan-tilt-zoom camera (PTZ camera) which is configured such that an imaging region is changeable. Furthermore, the camera 100 may correspond to the power over Ethernet (PoE) or may be configured such that electric power is supplied through a LAN cable. Furthermore, although the single camera 100 is connected to the network 300 in
The client apparatus 200 is configured by a terminal apparatus, such as a personal computer (PC), a smartphone, or a tablet PC, and is capable of transmitting various commands to the camera 100. Examples of the commands transmitted to the camera 100 include a command for changing an angle of view and an imaging direction (a direction) of the camera 100, a command for changing an imaging parameter, a command associated with image processing, and a command for starting transmission of an image. The client apparatus 200 may receive a response to such a command from the camera 100.
The camera 100 changes an imaging direction and an angle of view when receiving the command for changing an imaging direction and an angle of view from the client apparatus 200. Furthermore, the camera 100 transmits an image to the client apparatus 200 in accordance with a command for starting transmission of an image. The client apparatus 200 receives an image transmitted from the camera 100 in response to the command for starting transmission of an image described above, records the received image, and performs display control of displaying the image for a user of the client apparatus 200.
As illustrated in
First, a configuration of the camera 100 and functions of the units will be described.
The imaging unit 101 includes a lens and an imaging element, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD), and captures an image of a subject. Furthermore, the imaging unit 101 performs a process of converting an optical image formed on an imaging surface into a digital electric signal by photoelectric conversion. The image processor 102 performs predetermined image processing on the signal obtained through the photoelectric conversion performed by the imaging unit 101. Examples of the image processing include a white balance process, a sharpness process, and a gradation conversion process. Furthermore, the image processor 102 generates image data by performing compression encoding on the image which has been subjected to the image processing and outputs the generated image data to the system controller 103.
The system controller 103 outputs the image supplied from the image processor 102 to the communication unit 104 so as to transmit the image to the client apparatus 200. Furthermore, the system controller 103 analyzes a command supplied from the client apparatus 200 through the communication unit 104 and performs processing in accordance with the command. Specifically, the system controller 103 issues an instruction for changing an image processing parameter to the image processor 102 in accordance with the received command. Furthermore, the system controller 103 performs a setting of parameters of the subject detection process on the subject detection unit 105 and a setting of parameters of the passing detection process on the determination unit 107 in accordance with the received command.
The subject detection unit 105 performs the subject detection process of detecting an object (a human body, for example) in the image generated by the image processor 102 by analyzing the image. Specifically, the subject detection unit 105 reads setting information of the detection region stored in the detection region storage unit 111 and performs the subject detection process on the detection region in the image generated by the image processor 102.
The subject ID assigning unit 106 assigns an ID for identifying a subject detected in images consecutively obtained to a result of the detection of the subject detection process performed by the subject detection unit 105. Specifically, the subject ID assigning unit 106 performs a matching process between subjects detected in the images consecutively obtained based on positional information and size information of the subject detected by the subject detection unit 105. Thereafter, the subject ID assigning unit 106 assigns a common ID to subjects which match each other and generates and assigns a new ID to a subject which does not match any subject as a subject which is newly detected. A plurality of subjects detected in the image may be distinguished from one another by assigned subject IDs.
The determination unit 107 performs the passing detection process of determining whether the subject detected by the subject detection unit 105 has passed the detection line set in the image. Note that the determination may not be performed using a line in the passing detection process as long as a process of determining whether a subject has passed a predetermined position (a reference position) in an image is performed. First, the determination unit 107 stores a current position of the subject to which an ID has been assigned by the subject ID assigning unit 106 in a memory or the like, not illustrated. Then the determination unit 107 reads the setting information of the detection line stored in the detection line storage unit 109 and determines whether the subject has passed the detection line based on a past position and the current position of the subject. When the determination is affirmative, the determination unit 107 increments a passing count so as to count the number of subjects which have passed the detection line.
The detection line setting unit 108 sets the detection line to be used for detecting passing of the subject and stores setting information of the detection line in the detection line storage unit 109. Here, the setting information of the detection line includes at least one of a position, a length, and an inclination of the detection line in the image.
The detection region setting unit 110 sets a detection region in the image which is to be subjected to the subject detection process based on the setting information of the detection line newly stored in the detection line storage unit 109 and stores the setting information of the detection region in the detection region storage unit 111. In other words, the detection region setting unit 110 sets, in accordance with the detection line, a detection region in the image so as to include the detection line.
Second, a configuration of the client apparatus 200 and functions of the units will be described.
The system controller 201 of the client apparatus 200 incorporates an image processor which combines images supplied from the camera 100 through the communication unit 204 and performs display control of displaying the images supplied from the camera 100 in the display unit 202. Furthermore, the system controller 201 generates various commands in accordance with graphic user interface (GUI) operations performed by the user and outputs the generated commands to the communication unit 204 so as to transmit the commands to the camera 100.
The display unit 202 includes a monitor, such as a liquid crystal display (LCD), and displays an image obtained from the camera 100 and GUIs. The input unit 203 includes a pointing device, such as a keyboard or a mouse, and the user of the client apparatus 200 operates the input unit 203 to input various commands through the GUIs. In this embodiment, the user of the client apparatus 200 operates the input unit 203 to specify a detection line described above through one of the GUIs.
The communication unit 204 receives images and signals supplied from the communication unit 104 of the camera 100 through the network 300 and transmits the received information to the system controller 201. Furthermore, the communication unit 204 transmits the commands described above to the communication unit 104 of the camera 100 through the network 300 based on an instruction issued by the system controller 201.
As illustrated in
The external memory 14 stores various data, various information, and the like required for executing processes performed by the CPU 11 using the programs. The external memory 14 stores various data, various information, and the like obtained when the CPU 11 executes processes using the programs and the like. The imaging unit 15 corresponds to the imaging unit 101 of
Note that, in a case of the client apparatus 200, a hardware configuration corresponding to the display unit 202 of
The camera 100 transmits an image to the client apparatus 200 so as to perform display control for displaying the image in the display unit 202 of the client apparatus 200 and accepts a designation of the detection line in the image displayed in the display unit 202. The camera 100 obtains setting information of the specified detection line when the user specifies the detection line in the image displayed in the display unit 202 of the client apparatus 200, and sets a detection region to be subjected to the subject detection process based on the setting information of the obtained detection line.
The user may specify an arbitrary detection line 410 in the setting screen 400. By the passing detection process using the detection line 410, a subject 500 which has moved rightward in the image and which has passed the detection line 410 may be detected, for example. According to this embodiment, the camera 100 sets a detection region 420 which is suitable for the passing detection process using the detection line 410 based on the setting information of the detection line 410 so that a region to be subjected to the subject detection process is restricted to the detection region 420.
In the subject detection process, the camera 100 extracts a feature value of an object (a human body, for example) from the detection region 420 in the image and performs template matching by comparing the feature value with a dictionary generated in advance. Here, the camera 100 performs horizontal scanning from an upper left portion in the detection region 420 and further performs horizontal scanning from left to right on a next line so as to detect an object (a human body, for example) in the detection region 420. Specifically, the subject detection process is not performed in a region out of the detection region 420.
Note that a case where the camera 100 serving as an imaging apparatus operates as an information processing apparatus which sets a detection region is described in this embodiment. Note that the client apparatus 200 may operate as the information processing apparatus or a general PC or another device may operate as the information processing apparatus. This is true for a second embodiment described below.
Next, an operation of the camera 100 according to this embodiment will be described.
The camera 100 may realize various processes illustrated in
First, in S1, the camera 100 receives setting information of a detection line specified by the user operating the setting screen displayed in the display unit 202 of the client apparatus 200. Here, the setting information of the detection line includes information on a position, a length, and an inclination of the detection line as described above. In S2, the camera 100 executes a detection region generation process of generating a detection region to be subjected to the subject detection process based on the setting information of the detection line obtained in S1.
It is assumed that a position of the detection line 410 specified by the user is associated with a region including a person to be detected and a length L of the detection line 410 is associated with a body height h of the person to be detected. The camera 100 sets a rectangle region having the length L of the detection line 410 in an extending direction of the detection line 410 as the detection region 420. It is assumed further that an inclination of the detection line 410 is associated with a movement direction of the person to be detected, and the camera 100 sets a rectangle region having predetermined detection widths W1 and W2 in a direction orthogonal to the detection line 410 as the detection region 420.
It is assumed in this embodiment that a body height h of a general subject is 1.7 m and the length L of the detection line 410 corresponds to 2 m in the image. To appropriately detect passing, distances corresponding to three steps forward and backward of the subject relative to the detection line 410 are denoted by the detection widths W1 and W2. Assuming that a length of a stride of the subject is 0.9 m, distances of 2.7 m relative to the detection line 410 in the image correspond to the detection widths W1 and W2 in forward and backward portions relative to the detection line 410. Assuming that the length L of the detection line 410 is 2 m, the detection widths W1 and W2 in the forward and backward portions relative to the detection line 410 are 1.35 times larger than the length L of the detection line 410. In this way, the camera 100 sets lengths obtained by multiplying the length L of the detection line 410 by a predetermined magnification (1.35) as the detection widths W1 and W2.
Note that a length of the detection region 420 in the extending direction of the detection line 410 may be obtained by adding a predetermined margin to the length L of the detection line 410. Specifically, the camera 100 determines a rectangle region which includes the detection line 410, which has at least the length L of the detection line 410 in the extending direction of the detection line 410, and which has the predetermined detection widths W1 and W2 in the direction orthogonal to the detection line 410 as the detection region 420 relative to the detection line 410.
Furthermore, the detection widths W1 and W2 may not be the same widths, and the detection width W1 may be larger than the detection width W2 so that a subject to be detected in passing thereof may be reliably detected.
In S21, the camera 100 calculates the detection widths W1 and W2 of the detection region by multiplying the length L of the detection line by a predetermined magnification (1.35 in this embodiment) based on the length L of the detection line included in the setting information of the detection line. In S22, the camera 100 generates a rectangle region which includes the length L of the detection line in the extending direction of the detection line and which is wider by the detection widths W1 and W2 from the two sides of the detection line in the direction orthogonal to the detection line. Finally, in S23, the detection region setting unit 110 of the camera 100 sets the rectangle region generated in S22 as a detection region relative to the detection line.
Referring back to
In S4, the camera 100 generates a circumscribed rectangle surrounding the entire overlapping detection region and the process proceeds to S5. In S5, the camera 100 deletes all setting information of the overlapping detection region from the detection region storage unit 111. Furthermore, the camera 100 sets the circumscribed rectangle generated in S4 as a common setting region of the detection settings.
In S6, the camera 100 stores the setting information of the detection region set in S2 or S5 in the detection region storage unit 111 and the process is terminated.
As described above, the camera 100 of this embodiment obtains the setting information of the detection line which is set in the image and which is to be used in the subject detection process and sets a detection region in the image to be subjected to the subject detection process based on the obtained setting information of the detection line.
By this, the camera 100 may automatically set a detection region of a subject suitable for passing detection using a detection line based on setting information of the detection line. In this way, the camera 100 may appropriately set a detection region of the subject suitable for the passing detection setting without performing a user operation, and accordingly, the passing detection process may be performed with high accuracy without bothering the user.
Before the setting information of the detection line is obtained, the detection line setting unit 108 of the camera 100 causes the system controller 201 of the client apparatus 200 to display an image so as to accept a designation of the detection line in the image displayed in the display unit 202. The user may specify a plurality of points in the image using a pointing device, such as a keyboard or a mouse, included in the client apparatus 200 with reference to the image displayed in the display unit 202 of the client apparatus 200 so as to specify a detection line. The camera 100 may obtain positional information of the plurality of points specified by the user in the image and information on lengths and inclinations of line segments formed by the plurality of points specified by the user as the setting information of the detection line.
Accordingly, the detection line setting unit 108 of the camera 100 may appropriately obtain the setting information of the detection line specified by the user and the detection region setting unit 110 of the camera 100 may set an appropriate detection region relative to the detection line specified by the user.
Furthermore, the detection region setting unit 110 of the camera 100 determines a region which includes the detection line, which has at least the length L of the detection line in the extending direction of the detection line, and which has the predetermined detection widths W1 and W2 in a direction orthogonal to the detection line as a detection region to be subjected to the subject detection process based on the setting information of the detection line. Here, the detection widths W1 and W2 are obtained by multiplying the length L of the detection line by a predetermined magnification.
In this way, the camera 100 focuses the association between the detection line specified by the user and the information on the subject which is to be subjected to the passing detection so as to set a position and a size of the detection region. Accordingly, the camera 100 may set a detection region in which a subject which has been passed is appropriately detected and accuracy of the passing detection process may be appropriately improved.
The camera 100 executes the subject detection process at a certain frame rate determined in advance on images which are consecutive in time-series. To accurately count objects (human bodies, for example) which have passed the detection line by the passing detection process, the subject detection process is required to be performed at a frame rate of a certain level or more. For example, the subject detection process is required to be performed on 10 images per one second. However, since objects (human bodies) are detected by the template matching in the subject detection process, a long period of time is required for the process. Therefore, in a case where a large number of people are included in the image or in a case where a complicated background is difficult to be distinguished from people, the process may not be completed in real time. In this case, the subject detection process is interrupted and a next image is subjected to the subject detection process, and therefore, accuracy of the passing detection process is degraded.
Specifically, as illustrated in
In a case where the subject 500 to be subjected to the passing detection is included in the lower portion in the image 600, the detection process is performed only on subjects 510 included in an upper portion in the image 600 which does not relate to a result of the passing detection process but the detection process is not performed on the subject 500. As a result, the subject 500 may not be appropriately detected and accuracy of the passing count is degraded.
On the other hand, in this embodiment, an appropriate detection region may be set in accordance with a set detection line so that the detection process is not performed on a region which does not relate to the passing detection. Accordingly, a subject to be subjected to the passing detection may be appropriately detected by the subject detection process, and the passing detection process may be performed with high accuracy.
A second embodiment of the present disclosure will now be described.
The case where a detection region is set based on setting information of a detection line is described in the foregoing first embodiment. In the second embodiment, a case where a detection region is set taking a moving speed of a subject in an orthogonal direction relative to a detection line into consideration will be described.
To appropriately detect passing of a subject, the subject is required to be detected a certain number of times before and after the subject has passed a detection line. Therefore, in this embodiment, a camera 100 sets detection widths W1 and W2 of a detection region such that the numbers of times the subject is detected before and after the subject has passed the detection line are equal to or larger than the set number of times.
Specifically, as illustrated in
First, as denoted by a dotted line in
As described above, the detection region setting unit 110 of the camera 100 according to this embodiment sets a detection region to be subjected to the subject detection process based on setting information of a detection line and a moving speed of a subject. Accordingly, as the moving speed of the subject to be subjected to the passing detection is higher, a larger detection region may be set. Specifically, the camera 100 sets the detection widths W1 and W2 of the detection region such that the numbers of times the subject is detected before and after the subject has passed the detection line are equal to or larger than the set numbers of times based on the moving speed of the subject. Accordingly, the subject may be appropriately detected and passing through the detection line may be detected with high accuracy. Furthermore, the camera 100 may set the number of times before the passing which is larger than the number of times after the passing. In this case, the passing of the subject may be more reliably detected.
A third embodiment of the present disclosure will be described.
The case where a detection region is set based on setting information of a detection line is described in the foregoing first embodiment. In the third embodiment, a case where a detection region is set taking a size of a subject into consideration will be described.
As illustrated in
Therefore, as described above with reference to
Therefore, in this embodiment, the detection region setting unit 110 of the camera 100 sets a length of a detection region in the extending direction of the detection line 410 based on a size of the subject 500. Specifically, the detection region setting unit 110 of the camera 100 sets a detection region 424 including a length corresponding to a size (a body height h) of the subject 500 on a side of the upper end 411 of the detection line 410 as illustrated in
As described above, the camera 100 according to this embodiment sets a detection region to be subjected to the subject detection process based on setting information of a detection line and a size of a subject. In this way, a detection region of a size which is sufficient for detection of the subject to be subjected to the passing detection may be set. Specifically, the detection region setting unit 110 of the camera 100 sets a length of the detection region in the extending direction of the detection line based on the size of the subject. Accordingly, the subject may be appropriately detected and passing through the detection line may be detected with high accuracy.
Furthermore, the detection region setting unit 110 of the camera 100 may set a detection region having at least a length corresponding to a size of a subject on an upper end side of a detection line. In this case, even a subject which barely passes a detection line by a foot may be more appropriately detected by the subject detection process and the passing through the detection line may be detected. Modifications 10086j In the foregoing embodiments, the case where a detection line extends in a vertical direction in an image is described. However, the detection line may horizontally or obliquely extend in the image.
When a detection line 410 obliquely extends as illustrated in
Furthermore, although the case where the camera 100 executes the process of generating a detection region is described in the foregoing embodiments, the client apparatus 200 may execute the process of generating a detection region. In this case, as illustrated in
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2017-209131 filed Oct. 30, 2017, which is hereby incorporated by reference herein in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2017-209131 | Oct 2017 | JP | national |