This application claims the benefit of Taiwan application Serial No. 104131761, filed Sep. 25, 2015, the disclosure of which is incorporated by reference herein in its entirety.
The disclosure relates in general to a video indexing method and a device using the same, and more particularly to a video indexing method which creates video indexes according to representative object snaps and a device using the same.
Along with the increase in the density of monitoring systems, video recording has become an indispensable tool in the maintenance of law and order and is normally used after the happening of an event. However, as the density of video recorders continuously increases, it would be extremely time-consuming to manually filter a large volume of video data.
Video synopsis is a latest video indexing technology, which, through time condensation, largely reduces the redundant parts in time and space of the video data and allows the user to conveniently browse the video and intercept video data.
However, how to increase video indexing efficiency for video synopsis is still a prominent task for the industries.
The disclosure is directed to a video indexing method and a device using the same capable of extracting objects from the video data and condensing the video data into one or more video indexing images according to the representative object snap of each object. Thus, the user can quickly browse the video content, and the video indexing efficiency can be increased.
According to one embodiment, a video indexing method is provided. The video indexing method includes steps of: analyzing trajectory information of a plurality of objects in a video data to obtain a sequence of object snaps including a plurality of object snaps; generating a sequence of candidate object snaps by filtering off some of the object snaps according to the appearance differences between the object snaps; selecting a plurality of representative object snaps from the sequence of candidate object snaps; and generating a video indexing image by merging the selected representative object snaps into a background image.
According to another embodiment, a video indexing device is provided. The video indexing device includes an analysis unit, a filter unit, a determination unit and an indexing generation unit. The analysis unit is for analyzing trajectory information of a plurality of objects in a video data to obtain a sequence of object snaps including a plurality of object snaps. The filter unit is for filtering off some of the object snaps according to the appearance differences between the object snaps to generate a sequence of candidate object snaps. The determination unit is for selecting a plurality of representative object snaps from the sequence of candidate object snaps. The indexing generation unit is for merging the selected representative object snaps into a background image to generate a video indexing image.
According to an alternative embodiment, a non-transitory computer readable recording medium with built-in program is provided. After the computer has loaded in and executed the program, the computer can complete the video indexing method of the present disclosure.
The above and other aspects of the invention will become better understood with regard to the following detailed description of the preferred but non-limiting embodiment (s). The following description is made with reference to the accompanying drawings.
In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the disclosed embodiments. It will be apparent, however, that one or more embodiments may be practiced without these specific details. In other instances, well-known structures and devices are schematically shown in order to simplify the drawing.
Some implementations of the present disclosure are disclosed in a number of embodiments with detailed descriptions and accompanying drawings. It should be noted that the structures and contents of the implementations are for exemplary purpose only, not for limiting the scope of protection of the present disclosure. The present disclosure does not disclose all possible embodiments. Any person ordinary skilled in the technology field, without violating the spirit and scope of the present disclosure, will be able to make necessary changes and modifications to the structures of the embodiments to meet actual needs. The above changes and modifications are also applicable to the implementations not disclosed in the present disclosure. Moreover, designations common to the embodiments are used to indicate identical or similar elements.
Refer to
The video indexing device 100 mainly includes an analysis unit 102, a filter unit 104, a determination unit 106 and an indexing generation unit 108. These units can be realized by such as an integrated circuit, a circuit board, or at least one readable programming code read from the at least one memory device by the processing unit.
In step 202, the analysis unit 102 analyzes trajectory information of a plurality of objects of the video data VD to obtain a sequence of object snaps S1, including, for example, a plurality of object snaps. The source of the video data VD is such as a video file, a video recorder of a mobile device, a network video streaming (such as YouTube), a network video recorder or a depth-of-field video recorder.
The analysis unit 102 extracts trajectory information of the objects by using the object detection and tracking algorithms. Examples of the object detection algorithm include the Gaussian mixture model (GMM) method, the temporal median filter method and the nonparametric kernel density estimation (KDE) method. Examples of the object tracking algorithm include the mean shift method, the cam shift method and the particle filter method.
For example, the analysis unit 102 creates a background image not containing any objects, and then compares the difference in each pixel between an input image and the newly created background image. If the difference is larger than a threshold, then the pixel is determined as a variant pixel, or referred as a foreground. In an embodiment, the analysis unit 102 can detect variant pixels by using a motion detection method, such asGaussians mixture model (GMM), temporal median filter or nonparametric kernel density estimation (KDE). After the variant pixels in the frame are obtained, different objects in the foreground are marked for tracking objects.
After the object detection and tracking procedure is completed, the analysis unit 102 obtains a sequence of object trajectory in the video data VD and object snaps, and further sort the object snaps to generate the sequence of object snaps S1.
In step 204, the filter unit 104 filters off some of the object snaps according to the appearance differences between the object snaps to generate a sequence of candidate object snaps S2. For example, the filter unit 104 filters the object snaps whose degrees of similarity are larger than a similarity threshold off the sequence of object snaps S1 to generate a sequence of candidate object snaps S2. In embodiments, the degrees of similarity are calculated according to at least one of the factors including object appearance, distance, motion vector and life cycle.
In step 206, the determination unit 106 selects a plurality of representative object snaps OR1˜ORN from the sequence of candidate object snaps S2. Each of the representative object snaps OR1˜ORN corresponds to an object in the video data VD.
In step 208, the indexing generation unit 108 merges the representative object snaps OR1˜ORN into a background image to generate one or more video indexing images I. In an embodiment, the analysis unit 102 analyzes a plurality of image snaps sampled from the video data and extracts a plurality of candidate background images. Then, the indexing generation unit 108 further selects one of the candidate background images as a background image.
The one or more video indexing images I generated by the indexing generation unit 108 can be shown on a screen for the user to view and analyze. For example, the user can click a representative object snap of the video indexing image I to browse the video content of the corresponding object.
In an embodiment, the video indexing device 100 further includes a setting unit 110 for determining an object density K, which can be used for determining the density of representative object snaps added to the video indexing image. For example, the indexing generation unit 108 sequentially merges representative object snaps OR1˜ORN into a background image, and outputs a video indexing image I1 when the density of representative object snaps of the background image reaches the object density K. Meanwhile, the video indexing image I1 includes K representative object snaps (such as OR1˜ORK) corresponding to K objects. Then, the representative object snaps (such as ORK+1˜ORN) having not been added to the video indexing image I1 are added to another video indexing image I2, and other values of object density K can be done in the same manner. The setting unit 110, which can be realized by such as a human-machine interface, sets the value of object density K in response to an external operation.
By using the method as indicated in
Since the representative object snaps OR1, OR2, and OR3 are sampled from the trajectory corresponding to the object OB1, OB2, and OB3, the representative object snaps appearing on the same video indexing image may correspond to the object snaps sampled at different time points. As indicated in
Based on the method of adding the object snaps to the object representative object snap and/or the value of the object density K, the object snaps sampled at the same sampling time point may appear in different video indexing images. That is, the contents of different video indexing images are not restricted by the priority by which the objects appear. Let
In an embodiment, the video data VD is divided into a plurality of sub-segments, and respective video indexing image corresponding to each sub-segment is generated. Let the uninterrupted video data VD obtained from a monitor be taken for example. The uninterrupted video data VD can be divided into a plurality of sub-segments in the unit of is minutes. Then, the method illustrated in
The analysis unit 102 generates a sequence of object snaps S1 by sequentially arranging the object snaps related to the same object. In the sequence of object snaps S1 illustrated in
In an embodiment, a new candidate object snaps ci is selected and placed at position li of the video indexing image, and the target function satisfying the minimal merging space for the candidate object snaps ci and previous object snap cj is expressed as:
G(i)=arg minciΣiεQ′Ea(li∩lj) (Formulas 1)
Wherein, Ea (.) represents the cost of having collision when the candidate object snap is placed in the video indexing image; Q represents a set of all object snaps; Q′ represents a set of candidate object snaps, and Q′⊂Q. Each time when a new object snap is added to the video indexing image, a video indexing image with compact space is generated by using a local optimum. In another embodiment, a global optimum is added to the candidate object snap.
Suppose the overlapped rate function of a candidate object snap ci is defined as follows:
Wherein, Area (ci) represents the area of the candidate object snap ci on the video indexing image; thr_a represents an overlapped rate threshold of the area overlapped rate of an object snap. If the overlapped rate of a newly added object snap is smaller than the overlapped rate threshold thr_a, then the newly added object snap can be added to the video indexing image I (i) according to its placing position. Conversely, if the overlapped rate of the newly added object snap is not smaller than the overlapped rate threshold thr_a, then the newly added object snap will not be added to the video indexing image (i) but will wait for a better space position in the next video indexing image. In an embodiment, a global area threshold thr_b can be set for each video indexing image. If the total area occupied by the currently added candidate object snaps is larger than the global area threshold thr_b, this implies that the frame is compact enough, and a next video indexing image I (i+1) can be generated.
As indicated in
Since the object overlapped rate of the object snap (3, 1) of the object OB3 for the representative object snaps OR1, which has been added to the video indexing image I1 is smaller than the overlapped rate threshold, the object snap (3, 1) is selected as the object representative object snap OR3 of the object OB3 and shown on the same video indexing image I1 together with the representative object snap OR1.
The methods of adding representative object snaps of the present invention are not limited to the above exemplifications. Any time/space algorithms considering the area and/or placing position of a representative object snap when optimizing the object overlapped rate are within the spirit of the invention.
The present disclosure further provides a non-transitory computer readable recording medium with built-in program capable of completing the video indexing methods disclosed above after the computer loads in and executes the program.
It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments. It is intended that the specification and examples be considered as exemplary only, with a true scope of the disclosure being indicated by the following claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
104131761 | Sep 2015 | TW | national |