This application claims the priority benefit of Chinese Patent Application No. 201510463583.6, filed on Jul. 31, 2015 in the Chinese State Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference.
1. Field
The present disclosure relates to the field of image processing technologies and, in particular, to a video data analyzing method and apparatus and a vehicle parking lot monitoring system.
2. Description of the Related Art
As the number of private vehicles increases rapidly in recent years, parking has become a serious problem for both drivers and the government. For drivers, it usually takes a long time to find a valid parking lot; for authorities, the statistical information on public and chargeable parking lots will help to relieve the problem and make for better planning. What's more, an intelligent parking monitoring system will report the status change of each parking lot over the time, hence assist the administrator to manage the parking site better.
There are already a number of solutions provided for stated parking guidance system. Generally, they are classified as sensor-based solutions and camera-based solutions. The sensor-based solutions are most widely adopted due to low cost. However, detection precision is not sufficient enough for parking status monitoring, as well as being limited by environment conditions. For example, systems that utilize an induction coil are interfered with by severe weather (e.g. lightening) and neighboring sensors; and systems that utilize ultra-sound sensors are not suitable for outdoor implementation. Camera-based solutions are promising due to the popularization of large scale video surveillance systems. Compared with sensor-based solutions, they provide more video information and make it possible to support advanced functions, such as plate number recognition. And there are different video analysis technologies used for the purpose of monitoring parking status.
Most camera-based solutions will use one customized camera to cover 2-3 parking lots, thus the entire system will be very expensive due to hardware cost, and not alone it is not suitable for outdoor implementation. To reduce system cost, some solutions are to evaluate the number of entering and exiting vehicles within a certain area instead, but the detailed status of each parking lot is not be provided.
It should be noted that the above description of the background is merely provided for clear and complete explanation of the present disclosure and for an easy understanding by those skilled in the art. And it should not be understood that the above technical solution is known to those skilled in the art as it is described in the background of the present disclosure
Additional aspects and/or advantages will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the invention.
In order to solve the problem pointed out in the Background, this application provides a video data analyzing method and apparatus and a parking lot monitoring system, so as to monitor a large quantity of parking lots by using few hardware resources, and realize high detection precision under an all-weather condition.
According to a first aspect of the embodiments of the present disclosure, there is provided a parking lot monitoring system, including:
According to a second aspect of the embodiments of the present disclosure, there is provided a video data analyzing method, including:
According to a third aspect of the embodiments of the present disclosure, there is provided a video data analyzing apparatus, including:
An advantage of the embodiments of the present disclosure exists in that with the method, apparatus or system provided in this application, a large quantity of parking lots may be monitored by using few hardware resources, and high detection precision under an all-weather condition may be realized, which is extendable, and may support high-grade functions based on core technologies.
With reference to the following description and drawings, the particular embodiments of the present disclosure are disclosed in detail, and the principle of the present disclosure and the manners of use are indicated. It should be understood that the scope of the embodiments of the present disclosure is not limited thereto. The embodiments of the present disclosure contain many alternations, modifications and equivalents within the scope of the terms of the appended claims.
Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.
It should be emphasized that the term “comprises/comprising/includes/including” when used in this specification is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.
The drawings are included to provide further understanding of the present disclosure, which constitute a part of the specification and illustrate the preferred embodiments of the present disclosure, and are used for setting forth the principles of the present disclosure together with the description. It is obvious that the accompanying drawings in the following description are some embodiments of the present disclosure only, and a person of ordinary skill in the art may obtain other accompanying drawings according to these accompanying drawings without making an inventive effort. In the drawings:
Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below to explain the present disclosure by referring to the figures.
These and further aspects and features of the present disclosure will be apparent with reference to the following description and attached drawings. In the description and drawings, particular embodiments of the disclosure have been disclosed in detail as being indicative of some of the ways in which the principles of the disclosure may be employed, but it is understood that the disclosure is not limited correspondingly in scope. Rather, the disclosure includes all changes, modifications and equivalents coming within the terms of the appended claims.
This application provides a parking lot monitoring system.
In this embodiment, the video camera 101 is configured to capture videos, so that the analysis server 103 analyzes a status of a parking lot according to the videos captured by the video camera 101. In this embodiment, a camera type and the number of the video camera 101 are not limited in this embodiment, and which type of video camera and how many video cameras are used depend on an application scenario, and a pan tilt zoom (PTZ) video camera, a bullet video camera, or other types of video cameras may be used. Furthermore, by considering the planning of a parking site where the system to be implemented and deploying video cameras with proper height and angle, optimal coverage efficiency may be achieved. In this embodiment, if a target scenario is an outdoor parking site (with no a roof) as shown in
In this embodiment, the data memory 102 is configured to store video data from the video camera 101, and provide the data to the analysis server 103 for further analysis and retrieve. Considering the convenience of system building, a network video recorder (NVR) may be used as the data memory 102. Of course, this embodiment is not limited thereto, and the data memory 102 may also be embedded in the camera 101, and provide data to the analysis server 103 via a camera network; or the data memory 102 may also be embedded in the analysis server 103 or in the central manage system 104.
In this embodiment, the analysis server 103 is configured to analyze a status of each parking lot according to the video data obtained from the data memory 102, and output an analysis result to the central manage system 104, a detailed analysis process is going to be described below. In this embodiment, in order to analyze the status of the parking lot, initialization needs to be performed on deployment of the parking site implementing the parking lot monitoring system 100 of this embodiment, including determining overall planning of the parking site, such as how many video cameras are deployed, a position of each video camera, which parking lots are covered by each video camera, a position of each parking lot, and coordinates of each parking lot, etc. By the initialization, deployment of the whole parking site may be defined, and information (coordinates) on each parking lot area may be obtained, so as to provide basis for the analysis server 103 for analyzing the videos captured by the video camera. After the deployment of the parking site is determined, only one time of initialization is needed, and initialization needs not to be repeated only if the deployment of the parking site is not changed. And the initialization may be performed in the analysis server 103, and may also be performed in the central manage system 104. For example, the initialization may be performed by a module added in the analysis server 103 or the central manage system 104; of course, this embodiment is not limited thereto.
In this embodiment, the central manage system 104 is a controller of the whole parking lot monitoring system 100, and is configured to control operations of parts of the parking lot monitoring system 100. It consists of central manage system (CMS) software and a hardware platform supporting the software, functions of the CMS including:
In this embodiment, the information publishing device 105 is configured to publish status information on each parking lot, so as to improve the convenience of both drivers and parking administrators. Typically, such kind of information may be published via a light-emitting diode (LED) poster at an entrance of the parking area, or may be updated via administration software or a website, and a smart mobile phone application is also a viable method. in this embodiment, functions of the information publishing device 105 may be integrated into the central manage system 104; however, this embodiment is not limited thereto.
In this embodiment, as the initialization is performed on the deployment of the parking site, the information on each parking lot may be obtained from the video data captured by the video camera, and the analysis server 103 determines the status of each parking lot by analyzing the video data of each parking lot.
In this implementation, the preprocessing unit 301 is configured to perform image preprocessing on the video data of the current frame, such as noise reduction, and image enhancement, etc., for the convenience of subsequent analysis. On the premise that quality of the video camera is very high and resolution of the captured video data is also very high, the preprocessing unit 301 may be omitted.
In this implementation, a method of noise reduction includes but is not limited to the following two manners: one is a temporal filter, which calculates continuous frames to generate one representative frame Fr to remove random image noises, only the representative frame being used for next operations; and the other one is a Gaussian filter, which operates on current frame to do smoothing. The related art may be referred to for a detailed process.
And in this implementation, a method of image enhancement includes but is not limited to a hue saturation value (HSV) equalizer, which helps to enhance image details under weak light conditions, such as rainy weather or nighttime. The related art may be referred to for a detailed process.
In this implementation, the patch extracting unit 302 is configured to extract a patch of the parking lot in video data of a current frame. The patch may be bigger than a parking space or parking slot.
In this implementation, the patch of the parking lot refers to an image area including a parking lot and its peripheral regions. As shown in
Referring again to
W
patch
=W+W×0.5×2=2 W;
H
patch
=H+H×0.25×2=1.5 H.
where W is width of the parking lot area or space 401, Wpatch is patch width, H is height of the parking lot area or space 401, and Hpatch is patch height.
For practical process, while the parking lot is shown as an irregular quadrilateral in the image, the same policy may also be used to acquire a patch area for processing, description of which being omitted herein.
In this implementation, the judging unit 303 is configured to judge whether the status of the parking lot changes according to the patch of the parking lot in the video data of the current frame and a patch of the parking lot in video data of a previous frame. In this implementation, the judging unit 303 may perform the above judgment by comparing the patch of the parking lot in the current representative frame and the patch of the parking lot in the previous representative frame. If it is judged yes, that is, an event or an action occurs at the parking lot, the vehicle detecting unit 304 may be used to detect whether a vehicle parks on the parking lot; otherwise, the step of vehicle detection may be jumped over and a status of the parking lot may be directly determined, thereby the redundant time-consuming vehicle detection operations may be avoided, an efficiency of the whole process may be improved, and frequent occurrence of such occupation status may be prevented.
In this implementation, a judgment method of the judging unit 303 may include but is not limited to an edge detection method. For example, a canny-like edge detector is used to generate an edge image of a patch of a parking lot that is to be processed, and if a difference between an edge image of the current frame and that of the previous frame is greater than a threshold THchange, it is determined that the parking lot area changes.
In this implementation, the current representative frame refers to the current frame, that is, a frame that is being currently processed, and the previous representative frame refers to a frame that was previously processed. It should be noted that as this embodiment is focused on a change of a status rather than tracking a whole process of an event, analysis on each frame of the video data is not high in efficiency. A part of frames of the video data are analyzed in this embodiment, the previous representative frame refers not to a previous frame of the current frame, but a frame that was previously processed before the current frame. As shown in
In this implementation, the vehicle detecting unit 304 is configured to detect whether a vehicle parks on the parking lot. In this implementation, the vehicle detecting unit 304 may detect the vehicle by using a feature detection method which is based on machine learning, including performing off-line training by using collected data set and performing on-line detection in a range of the patch of the parking lot. For such operations, an improving algorithm featured with a histogram of gradient (HoG) may be used; however, other features, such as a Harr-like feature, may also be used for such operations. And once a vehicle is detected, a target rectangle corresponding to the vehicle may be obtained.
In this implementation, the status determining unit 305 is configured to determine a status of the current parking lot according to a judgment result of the judging unit 303 and/or a detection result of the vehicle detecting unit 304, and provide the status to the central manage system 104. As described above, if it is judged by the judging unit 303 that the status of the current parking lot does not change, the status determining unit 305 determines that the status of the current parking lot as being the previous status, and if it is judged by the judging unit 303 that the status of the current parking lot changes and it is detected by the vehicle detecting unit 304 that no vehicle parks on the parking lot, the status determining unit 305 determines that the status of the current parking lot as being not occupied; otherwise, the status determining unit 305 determines the status of the current parking lot according to a relationship between the detected target rectangle corresponding to the vehicle and the area of the current parking lot.
For example, if a ratio (Rveh) of a distance (Ldist) from a central point of the target rectangle to any side of the parking lot in a first direction to a length (Llot) of the parking lot in the first direction is greater than a first threshold, the status of the parking lot is determined as being normally occupied, otherwise, the status of the parking lot is determined as being illegally occupied.
If Rveh is greater than a predetermined threshold THwarning the status determining unit 305 determines that the status of the current parking lot as being normally occupied; otherwise, it shows that the vehicle is carelessly parked between two parking places or slots, and the status determining unit 305 determines that the status of the current parking space or slot as warning. And at this moment, the central manage system 104 may notify the administrator by sending a message. In this example, x denotes a horizontal coordinate direction.
With the analysis server of this embodiment, a status of each parking space may be determined, thereby avoiding redundant time-consuming vehicle detection operations, improving an efficiency of the analysis.
In this embodiment, an object of the parking lot monitoring system 100 is to analyze a status of a parking lot when an event of vehicle parking or vehicle leaving occurs. As this embodiment focuses on a change of a status rather than tracking a whole process of an event, when it is implemented as a commercial solution, analysis on data of each frame of the video stream of a video camera is inefficient. In this embodiment, a polling mechanism is proposed to assist the analysis server 103 in analyzing video streams of multiple video cameras.
In this embodiment, one analysis server 103 performs orderly or ordered analysis on video data captured by multiple video cameras in one analysis process using a temporal order, video data captured by each video camera being stored in the same buffer.
Hence, assuming that N analysis processes are running simultaneously on an analysis server 103 and each analysis process handles video data from K video cameras, then an analysis server 103 works for N×K video cameras; wherein, both N and K depend on a hardware capacity of the server and the number of parking slots or spaces covered by one video camera.
With the parking lot monitoring system of this embodiment, a large quantity of parking lots may be monitored by using few hardware resources, and high detection precision under an all-weather condition may be realized, which is extendable, and may support high-grade functions based on core technologies, such as calculation of a parking time, warning of illegal parking, and classification of vehicle types, etc.
This application further provides a video data analyzing method. As principles of the method for solving problems are similar to that of the analysis server 103 in Embodiment 1, the implementation of the analysis server 103 in Embodiment 1 may be referred to for implementation of the method, with identical contents being not going to be described any further.
In step 704, if it is detected that no vehicle parks on the parking lot, it is determined that the status of the parking lot as being not occupied, and if it is detected that a vehicle parks on the parking lot, whether the status of the parking lot is being normally occupied or illegally occupied is determined according to a relationship between a target rectangle corresponding to the vehicle and an area of the parking lot.
In this embodiment, if a ratio of a distance from a central point of the target rectangle to any side of the parking lot in a first direction to a length of the parking lot in the first direction is greater than a first threshold, the status of the parking lot is determined as being normally occupied, otherwise, the status of the parking lot is determined as being illegally occupied.
In this embodiment, the method may further include:
In this embodiment, a method of vehicle detection is not limited, and details are as described above. And when it is detected that a vehicle parks on the parking lot, the method further includes: acquiring a target rectangle corresponding to the vehicle.
With the method of this embodiment, a large quantity of parking lots may be monitored by using few hardware resources, and high detection precision under an all-weather condition may be realized, which is extendable, and may support high-grade functions based on core technologies, such as calculation of a parking time, warning of illegal parking, and classification of vehicle types, etc.
This application further provides a video data analyzing apparatus. As principles of the apparatus for solving problems are similar to those of the analysis server 103 in Embodiment 1, the implementation of the analysis server 103 in Embodiment 1 may be referred to for implementation of the apparatus, with identical contents being not going to be described any further.
With the method of this embodiment, a large quantity of parking slots or spaces may be monitored by using few hardware resources, and high detection precision under an all-weather condition may be realized, which is extendable, and may support high-grade functions based on core technologies, such as calculation of a parking time, warning of illegal parking, and classification of vehicle types, etc.
An embodiment of the present disclosure further provides a computer system, including the video data analyzing apparatus described in Embodiment 3, and the video data analyzing apparatus may be carried out by the analysis server 103 in Embodiment 1. As the analysis server 103 has been described in detail in Embodiment 1, the contents of which are incorporated herein, and shall not be described herein any further.
In an implementation, the functions of the analysis server 103 described in Embodiment 1 may be integrated into the central processing unit 801.
In another implementation, the analysis server 103 described in Embodiment 1 and the central processing unit 801 may be configured separately. For example, the analysis server 103 may be configured as a chip connected to the central processing unit 801, with its functions being realized under control of the central processing unit 801.
As shown in
As shown in
In this embodiment, the memory 802 may be, for example, one or more of a buffer memory, a flash memory, a hard drive, a mobile medium, a volatile memory, a nonvolatile memory, or other suitable devices. The memory 802 may store predefined or preconfigured information, video data captured by a video camera, and may further store programs executing related information. And the central processing unit 801 may execute the programs stored in the memory 802, so as to realize information storage or processing, etc. Functions of other parts are similar to those of the related art, which shall not be described herein any further. The parts of the computer system 800 may be realized by specific hardware, firmware, software, or any combination thereof, without departing from the scope of the present disclosure.
In this embodiment, the functions of the data memory 102 and the functions of the data buffer of Embodiment 1 may be integrated into the memory 802, and the functions of the central manage system 104 and the functions of the information publishing device 105 may be integrated into the central processing unit 801.
With the computer system 800 of this embodiment, a large quantity of parking slots may be monitored by using few hardware resources, and high detection precision under an all-weather condition may be realized, which is extendable, and may support high-grade functions based on core technologies, such as calculation of a parking time, warning of illegal parking, and classification of vehicle types, etc.
An embodiment of the present disclosure further provides a computer-readable program, wherein when the program is executed in a computer, the program enables the computer to carry out the method as described in Embodiment 2.
An embodiment of the present disclosure further provides a non-transitory storage medium in which a computer-readable program is stored, wherein the computer-readable program enables a computer to carry out the method as described in Embodiment 2.
The above apparatuses and methods of the present disclosure may be implemented by hardware, or by hardware in combination with software. The present disclosure relates to such a computer-readable program that when the program is executed by a logic device, the logic device is enabled to carry out the apparatus or components as described above, or to carry out the methods or steps as described above. The present disclosure also relates to a non-transitory storage medium for storing the above program, such as a hard disk, a floppy disk, a CD, a DVD, and a flash memory, etc.
The present disclosure is described above with reference to particular embodiments. However, it should be understood by those skilled in the art that such a description is illustrative only, and not intended to limit the protection scope of the present disclosure. Various variants and modifications may be made by those skilled in the art according to the principle of the present disclosure, and such variants and modifications fall within the scope of the present disclosure.
Although a few embodiments have been shown and described, it would be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the disclosure, the scope of which is defined in the claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
201510463583.6 | Jul 2015 | CN | national |