METHOD FOR TRACKING OBJECT IN DYNAMIC SCENE AND SYSTEM THEREOF

Abstract

A method for tracking an object in a dynamic scene and a system are provided. The object-tracking system acquires continuous frames and identifies one or more objects in each of the frames. A target object is chosen from a first frame. An object distance between the target object and each of the objects identified in a second frame is calculated. Word strings on the target object and every object identified in the second frame are recognized. A string similarity between the string on the target object and the string on each of the objects in the second frame is calculated. An overall score of the object distance and the string similarity is calculated and is referred to for determining if the target object is present in the second frame. Object-tracking is implemented when the target object is determined to be present in the second frame and subsequent frames.

Description

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application claims the benefit of priority to patent application No. 202311412481.2, filed on Oct. 27, 2023, in the People's Republic of China. The entire content of the above identified application is incorporated herein by reference.

Some references, which may include patents, patent applications and various publications, may be cited and discussed in the description of this disclosure. The citation and/or discussion of such references is provided merely to clarify the description of the present disclosure and is not an admission that any such reference is “prior art” to the disclosure described herein. All references cited and discussed in this specification are incorporated herein by reference in their entireties and to the same extent as if each reference was individually incorporated by reference.

FIELD OF THE DISCLOSURE

The present disclosure relates to an object-tracking method, and more particularly to a method for tracking an object in a dynamic scene according to an object distance and a string similarity between a previous frame and a next frame, and a system for performing the same.

BACKGROUND OF THE DISCLOSURE

Currently, a multi-object tracking algorithm performed on continuous images is used to determine whether or not the objects in different frames (e.g., a previous frame and a next frame) are a same object, or any new object is found in the following frames. The algorithm is used to calculate a matching degree between the objects respectively present in the previous frame and in the next frame. The objects present in different frames are determined to be the same object and can be assigned with a same object number if the matching degree between the objects in the different frames is high enough. Otherwise, the object present in the next frame is determined to be a new object and is assigned with a new object number if the matching degree between the objects in the different frames is lower than a threshold.

The conventional methods for calculating the matching degree for objects are such as an intersection over union (IoU) method (e.g., Sort algorithm) that calculates a matching degree according to an overlapping proportion between two objects respectively in a previous frame and in a next frame, and an appearance metric method (e.g., DeepSort algorithm) that calculates an appearance degree between two objects.

However, the above-mentioned conventional methods may encounter some problems if the methods are applied to for recognizing a license plate in a dynamic scene. One of the problems is that, since a relative speed of an oncoming car compared to a camera device used to calculate the matching degree is high, a moving distance of the license plate of the same car in the two adjacent frames is large and the images of the license plate of the same car in the two adjacent frames are not overlapped. Therefore, it is difficult to determine whether or not the same object appears in both the previous and next frames based on an overlapping level. The other problem is that the license plates generally have a light background with dark characters in appearance and have similar outlines and aspect ratios, and the difference of characters of the license plates is insufficient to be the appearance features for distinguishing the difference license plates. Accordingly, the appearance feature of the license plate cannot be a reference alone to determine whether or not two license plates are the same license plate. One further problem is that, in a circumstance that dense cars are parked on the roadside or a coming car on an opposite lane of highway can be seen, it often happens that many license plates may appear at the same position in multiple continuous images within a short period of time. Therefore, even though the objects appear at the same position of multiple continuous frames, it still cannot determine that the objects are the same object.

For solving the above problems, an advanced license plate extraction process is required. However, if the device that is required to calculate the matching degree is with an embedded system, the device cannot provide sufficient computing power to extract features of each of the license plates from every frame.

SUMMARY OF THE DISCLOSURE

In response to the above-referenced technical inadequacies that the license-plate recognition technology cannot only rely on the position of the license plate to determine if the same license plate appears in different frames, the present disclosure provides a method for tracking an object in a dynamic scene and a system. In the method, when taking the position of the license plate into consideration, it does not require that the license plates to be determined as the same license plate need to be overlapped in continuous frames in the dynamic scene, and the overlapped license plates in a previous frame and in a next frame will not be directly determined as the same license plate. The method can be applied to the dynamic scene, by which a relative distance between two objects (e.g., the license plates) in the previous frame and the next frame is obtained and assigned with a distance score. Further, a similarity between the two objects in the previous frame and the next frame is calculated. For license plate recognition, a string similarity between the strings of the license plates is calculated according to a result of license plate recognition and assigned with a similarity score. The system performs object recognition and tracking according to the distance score and the similarity score.

According to one embodiment of the object-tracking system of the present disclosure, the system includes a microcontroller, an image processor and a photographing module. The image processor is used to perform object recognition and object positioning, the photographing module is used to take a video and the microcontroller performs the object-tracking method.

In the method for tracking an object in a dynamic scene, a photographing module is used to take a video including continuous frames. A previous frame and a next frame are represented by a first frame and a second frame respectively. An image processor frame-by-frame recognizes one or more objects in the frames. One of the objects in the first frame is selected as a target object. Next, one or more distances between the target object of the first frame and one or more objects recognized from the second frame are calculated. The one or more distances are such as one or more object distances. After the strings of the target object of the first frame and the one or more objects of the second frame are recognized. A string similarity between the string of the target object of the first frame and each of the one or more objects of the second frame is calculated.

In the method, an object distance between the target object of the first frame and each of the objects of the second frame and a string similarity are referred to for calculating an overall score. The object-tracking system relies on the overall score to determine whether or not the target object in the first frame is any object appearing in the second frame.

Preferably, the object distance is a distance between a geometric center of the target object in the first frame and the geometric center of each of the one or more objects in the second frame.

Preferably, both string of the target object of the first frame and string of each of the objects of the second frame are license-plate numbers, each of which is a combination of multiple English alphabets and/or numbers; and the string similarity between the strings of two objects is an edit distance.

Further, when the overall score between the target object of the first frame and each of the objects of the second frame is obtained, the overall score is compared with a score threshold so as to determine whether or not the target object of the first frame appears as any object in the second frame according to a comparison result.

Accordingly, when it is determined that the target object appears in multiple continuous frames based on the overall score, a unique identifier is assigned to the target object in the continuous frames for tracking the target object.

These and other aspects of the present disclosure will become apparent from the following description of the embodiment taken in conjunction with the following drawings and their captions, although variations and modifications therein may be affected without departing from the spirit and scope of the novel concepts of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The described embodiments may be better understood by reference to the following description and the accompanying drawings, in which:

FIG. 1A and FIG. 1B are schematic diagrams depicting circumstances that a method for tracking object in dynamic scene is performed;

FIG. 2 is a schematic diagram illustrating a relationship of license plates in a previous frame and in a next frame in the method for tracking object in dynamic scene;

FIG. 3 is a schematic diagram illustrating main function modules of a dynamic scene object-tracking system according to one embodiment of the present disclosure;

FIG. 4 is a flowchart illustrating the method for tracking object in dynamic scene according to one embodiment of the present disclosure;

FIG. 5 is a flowchart illustrating a process of tracking a license plate according to one embodiment of the present disclosure;

FIG. 6 is a schematic diagram illustrating a process of determining the same license plate in continuous frames according to one embodiment of the present disclosure;

FIG. 7 is a flowchart illustrating a process of determining the same license plate in continuous frames in the method for tracking object in dynamic scene according to one embodiment of the present disclosure; and

FIG. 8 is a flowchart illustrating a process of recognizing characters of string of license plate in one embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

The present disclosure is more particularly described in the following examples that are intended as illustrative only since numerous modifications and variations therein will be apparent to those skilled in the art. Like numbers in the drawings indicate like components throughout the views. As used in the description herein and throughout the claims that follow, unless the context clearly dictates otherwise, the meaning of “a,” “an” and “the” includes plural reference, and the meaning of “in” includes “in” and “on.” Titles or subtitles can be used herein for the convenience of a reader, which shall have no influence on the scope of the present disclosure.

The terms used herein generally have their ordinary meanings in the art. In the case of conflict, the present document, including any definitions given herein, will prevail. The same thing can be expressed in more than one way. Alternative language and synonyms can be used for any term(s) discussed herein, and no special significance is to be placed upon whether a term is elaborated or discussed herein. A recital of one or more synonyms does not exclude the use of other synonyms. The use of examples anywhere in this specification including examples of any terms is illustrative only, and in no way limits the scope and meaning of the present disclosure or of any exemplified term. Likewise, the present disclosure is not limited to various embodiments given herein. Numbering terms such as “first,” “second” or “third” can be used to describe various components, signals or the like, which are for distinguishing one component/signal from another one only, and are not intended to, nor should be construed to impose any substantive limitations on the components, signals or the like.

In response to the technical inadequacies that the conventional license-plate recognition algorithm is not suitably to be applied to a scene that is dynamically changed for tracking a specific object, the present disclosure provides a method for tracking object in dynamic scene and a system. The method neither requires that the license plates in continuous frame should be overlapped, nor relies on the overlapped license plates in a previous frame and in a next frame to determine the same license plate. Instead, the method obtains a relative distance of two objects (e.g., the license plates) respectively in the previous frame and in the next frame, and assigns the distance a distance score. On the other hand, a similarity between every two objects respectively in the previous frame and in the next frame is calculated. For example, the similarity between two objects can be a string similarity between two license plates. A string similarity between the two license plates respectively in the previous frame and in the next frame is then calculated and is assigned with a similarity score. The further object recognition and tracking can be performed according to the distance score and the similarity score.

References are made to FIG. 1A and FIG. 1B, which are schematic diagrams illustrating the scenes where the method for tracking object in dynamic scene is performed in one embodiment of the present disclosure.

FIG. 1A shows a first frame 101 selected from continuous frames captured by a photographing module of a terminal device (e.g., a drive recorder). The first frame 101 shows vehicles 11, 12 and 13 in front of a rear vehicle having the terminal device. An object-detection model operated in the terminal device can be used to identify a license plate 110 of the vehicle 11, a license plate 120 of the vehicle 12 and a license plate 130 of the vehicle 13 from the first frame 101. The vehicle 11 is selected as a target vehicle and the license plate 110 of the vehicle 11 is selected as a target license plate.

FIG. 1B shows a second frame 102 selected from the continuous frames. The second frame 102 still has the target vehicle 11 and the image of the license plate 110 is still taken. In an edge-computing method for automatic license plate recognition, a license plate recognition technology is applied to the continuous frames for tracking the vehicle 11 having the license plate 110. In the present example, according to a result of license plate recognition, the second frame 102 still shows the vehicle 11, but does not include the vehicles 12 and 13 that are parking on a roadside. Another vehicle 14 having a license plate 140 parking on the roadside is found in the second frame 102.

The system uses the photographing module to take the continuous frames and tracks the license plate that appears in the continuous frames. The photographing module can be the drive recorder installed inside a moving vehicle. The object-tracking system of the present disclosure is suitably used in an electronic device installed in the moving vehicle. Therefore, a purpose of tracking object in a dynamic scene can be achieved.

Reference is made to FIG. 2, which is a schematic diagram depicting a relationship of two license plates respectively appearing in a previous frame and in a next frame in the method for tracking object in dynamic scene.

FIG. 2 shows a first frame 21 selected from the continuous frames. Several license plates 211, 212 and 213 are recognized from the first frame 21. In the method for tracking object, the license plate 211 with a license plate number “0001” is selected as a target license plate.

The object-tracking system extracts another frame (e.g., the second frame 22) from the continuous frames. Multiple license plates 221, 222 and 223 are recognized from the second frame 22 by an object recognition process. In the object-tracking method, distances between the target license plate 211 of the first frame 21 and each of the license plates of the second frame 22 can be calculated. The present example shows that the position of the target license plate 211 in the first frame 21 corresponds to the position of the target license plate 211′ with a license plate number “0001” of the second frame 22. Since the scene is dynamically changed, the position of the target license plate 211 can be changed to be the position of one of the license plates 221, 222 and 223 of the second frame 22.

For determining whether or not the target license plate 211 of the first frame 21 is any license plate of the second frame 22, multiple distances between the target license plate 211 and each of the license plates 221, 222 and 223 recognized from the second frame 22 can be calculated. Based on the target license plate 211′ that corresponds to the position of the target license plate 211 of the first frame 21 labeled in second frame 22, a first object distance d1 between the target license plate 211′ and the license plate 221, a second object distance d2 between the target license plate 211′ and the license plate 222, and a third object distance d3 between the target license plate 211′ and the license plate 223 are calculated. The distance to be calculated can be a straight-line distance between two objects in the image, and the object distance is a distance between a geometric center of the target object in the first frame and the geometric center of each of the one or more objects in the second frame.

After one or more object distances between the selected target license plate 211 and the one or more license plates recognized from another frame (e.g., a next frame or any frame before or after the current frame) is obtained, the one or more object distances acts as one of basis for determining whether or not the target license plate 211 appears in another frame. The present example shown in FIG. 2 shows that, in the continuous frames taken in a rapidly changing scene, the target license plate 211 with a license plate number “0001” being selected from the first frame 21 becomes the license plate 221 shown in the second frame 22. The license plate 221 has been recognized as the license plate with the license plate number “0001.” That means the target license plate 211 has been recognized and tracked in the continuous frames. According to the certain embodiments of the present disclosure, the object-tracking system can conduct recognition and tracking on multiple target license plates simultaneously.

It should be noted that, for the purpose of object tracking, in addition to calculating the object distance between the target object (e.g., the target license plate) and one or more objects in another frame for recognizing the same object in a dynamic scene, a similarity (e.g., a string similarity) of features between the target object and the object in a different frame is also taken into consideration for recognizing the same object.

Taking the license plate as a main embodiment, it cannot only rely on appearance of license plate to determine whether or not the objects in a previous frame and in a next frame are the same object since the appearances of general license plates are similar. Therefore, similarity of features of the objects (e.g., license plates) is taken into consideration. For example, the similarity is such as a string similarity since a string of a general license plate is a license plate number that is a combination of multiple English alphabets and/or numbers. The system will consider the string similarity between string of the target object of the first frame and string of each of the objects of the second frame as a reference to determine if the object appears in the adjacent frames. In an exemplary example, a Levenshtein distance that is a kind of edit distance is used to calculate a minimum number of single-character edits between two strings when one of the strings is converted to another string. The minimum number of single-character edits can be construed as the edit distance between two strings. The object-tracking method of the present disclosure is performed based on the string similarity of the license plates respectively in the previous frame and in the next frame. After the string similarity is calculated, the two strings are identical if the edit distance is “0” or the two strings are different if the edit distance is “1.” The edit distance acts as the string similarity and is represented between “0” and “1” since the minimum number of single-character edits is calculated and normalized. The string similarity can effectively exclude the problems that the license plate may not be recognized accurately due to lights, shadows and various factors.

Calculation of the Levenshtein distance (lev_a,b(i,j)) refers to equation 1. In equation 1, “|a|” and “|b|” represent lengths of the strings “a” and “b.” The Levenshtein distance between these two strings is “lev_a,b(|a|,|b|)” that can be obtained by equation 1. “lev_a,b(i,j)” denotes the Levenshtein distance between first “i” characters of “a” and first “j” characters of “b.”

$\begin{array}{cc}{\mathrm{lev}}_{a,b}\left(i,j\right)=\{\begin{array}{c}\max \left(i,j\right)\mathrm{if}\min \left(i,j\right)=0,\\ \min \{\begin{array}{c}{\mathrm{lev}}_{a,b}\left(i-1,j\right)+1\\ {\mathrm{lev}}_{a,b}\left(i,j-1\right)+1\\ {\mathrm{lev}}_{a,b}\left(i-1,j-1\right)+1_{\left(a_i\ne b_j\right)}\end{array}\mathrm{otherwise}\end{array}.& \mathrm{Equation}1\end{array}$

In the method for tracking object in dynamic scene, both calculation results of the object distance and the edit distance (i.e., the string similarity) are taken into consideration. According to an actual condition of the dynamic scene, for example, a moving vehicle mounting an electronic device operating the object-tracking system, a specific ratio (e.g., 4:6 or any ratio including 1:1) can be applied to between the object distance and the edit distance for calculating an overall score that is used to determine whether or not the objects respectively in the previous frame and in the next frame are the same object. The overall score is then compared with a determination threshold set by the object-tracking system so as to confirm the same object in the continuous frames. The objects determined to be the same object are assigned with an object identifier (e.g., a number).

Reference is made to FIG. 3, which is a schematic diagram illustrating main function modules of the object-tracking system in a dynamic scene according to one embodiment of the present disclosure. The various function modules that are implemented through collaboration of software and hardware circuits perform object tracking based on the object distance and the edit distance. In one of the embodiments, the main function modules of the object-tracking system include a photographing module 31, a license-plate-recognition module 33, a computation module 35 and a memory unit 37 that is used to buffer the processed data (e.g., data of the continuous frames and license plates) and the values (e.g., the object distances, edit distances and overall scores) generated by various calculations.

According to one embodiment of the present disclosure, the license-plate-recognition module 33 is implemented by an image processor of the electronic device. The license-plate-recognition module 33 performs object recognition and object positioning through collaboration of firmware, software and/or hardware. In one embodiment, the license-plate-recognition module 33 performs a license-plate-recognition procedure 331 and a string-recognition procedure 333. The computation module 35 can be implemented by a microcontroller of the electronic device. The computation module 35 primarily performs an object-distance calculation program 351, a string-similarity calculation program 353 and a license-plate tracking program 355.

According to the embodiment of the function modules of the object-tracking system shown in FIG. 3, an operating procedure of the system can refer to FIG. 2, which is a schematic diagram depicting a relationship between license plates respectively in the previous frame and in the next frame and FIG. 4, which is a flowchart illustrating the method for tracking object in dynamic scene according to one embodiment of the present disclosure.

In the flowchart of the method for tracking object in dynamic scene, the photographing module 31 of the object-tracking system is used to take a video and extract continuous frames (step S401). The license-plate-recognition module 33 performs the license-plate-recognition procedure 331 for frame-by-frame recognizing one or more license plates in each of the frames (step S403). For example, the first frame 21 of FIG. 2 includes a target license plate 211 and other license plates 212 and 213, and the second frame 22 includes license plates 221, 222 and 223. It should be noted that the first frame 21 having the target license plate 211 is a currently-received image and the second frame 22 indicates any of other frames in the continuous frames. In other words, the second frame 22 can be any frame before or after the first frame 21.

In the schematic diagram shown in FIG. 2, after a target object (e.g., the target license plate 211) is selected, the computation module 35 that is implemented by a central processor and related circuits of the object-tracking system performs the object-distance calculation program 351 for calculating one or more object distances between the position of the target license plate 211 of the first frame 21 and one or more objects (e.g., the license plates 221, 222 and 223) of the second frame 22 (step S405).

Afterwards, the string-recognition procedure 333 operated in the license-plate-recognition module 33 is used to recognize strings of the target object of the first frame (e.g., target license plate of the first frame 21 of FIG. 2) and one or more objects of the second frame (e.g., license plates 221, 222 and 223 of the second frame 22 of FIG. 2) (step S407). The technology for recognizing characters of the string of the license plate can refer to the flowchart of recognizing string of the license plate in FIG. 8.

The computation module 35 performs the string-similarity calculation program 353 so as to calculate a string similarity between two objects. The string similarity can be embodied by the Levenshtein distance. Calculation of the edit distance is to obtain the string similarity between the strings of the target object of the first frame and each of the objects of the second frame, namely the string similarity between two license plates (step S409).

After the object distance between the target object of the first frame and each of the one or more objects of the second frame and the string similarity between every two objects are obtained, a distance score is assigned to the object distance and a string-similarity score is assigned to the string similarity, an overall score can be calculated based on the distance score and the string-similarity score (step S411). In an aspect, an average of the distance score and the string-similarity score can be the overall score. In another aspect, a ratio between the distance score and the string-similarity score can be set when taking an actual dynamic scene into consideration, and the ratio referred to for calculating the overall score.

A comparison result of comparing the overall score and a score threshold set by the system is used to determine whether or not the target object in the first frame is any object appearing in the second frame. For example, the object having the overall score larger than or equal to the score threshold in the second frame is regarded as the target object in the first frame and the objects are assigned with the same identifier (e.g., a number) (step S413). After the same objects are determined in the continuous frames, the license-plate tracking program 355 performed by the computation module 35 can track the license plate. Reference is made to FIG. 5, which is a flowchart illustrating a process of tracking a license plate according to one embodiment of the present disclosure.

When the license plates in each of the frames are recognized, the license plates are one-by-one numbered. The overall score being obtained according to the distance score and the string-similarity score between every two license plates in different frames is referred to for determining whether or not the same license plate appears in the continuous frames (step S501). If the overall score is smaller than a score threshold set by the system, the license plates in the continuous frames are determined to be different license plates (determination is “no”). That is determined that a license plate of a new vehicle appears, and the system assigns a new identifier to the new license plate (step S503).

Otherwise, if the overall score is larger than the score threshold, it is determined that the target license plate of the first frame appears in other frames (determination is “yes”), and the same identifier that can be newly-added or modified is assigned to the target license plate in the continuous frames based on the overall score (step S505). The license plate can be tracked according to the same identifier in the continuous frames (step S507).

A unique identifier can be assigned to the same license plate found in the continuous frames, and the unique identifier assigned to the license plate can be marked on the picture displayed on a display of the electronic device. The license plate numbers that have been recognized can also be marked on the picture for a user to monitor the image of the vehicle or the license plate that is recognized from the continuous frames. The identifier or the license plate number displayed on the display of the electronic device is provided for easily tracking the vehicle.

In addition to the above description, here are some scenarios that the above-described method will encounter. In one of the scenarios, when the overall scores between the target license plate of the first frame and all license plates of the second frame are obtained, if the overall scores between multiple license plates of the second frame or other frames and the target license plate of the first frame are larger than or equal to the score threshold, the license plate having the highest overall score can be regarded as the same with target license plate of the first frame, and is assigned with the same identifier (e.g., a number) with the target license plate.

However, in one further scenario, when both the first frame and the second frame have multiple license plates, there will be an exception if only the license plate having the highest overall score is determined as the same license plate with the target license plate. The method for recognizing the characters of the string of the license plate of the present disclosure is described as follows.

Reference is made to FIG. 6, which is a schematic diagram showing multiple objects (e.g., multiple license plates) to be recognized respectively from a first frame 61 and a second frame 62 of the continuous frames according to one embodiment of the present disclosure. The method for tracking object in dynamic scene is used to determine the same license plate from the continuous frames so as to track the license plate. FIG. 7 is a flowchart illustrating the method for tracking object in dynamic scene according to one embodiment of the present disclosure.

FIG. 6 schematically shows three license plates 1A, 1B and 1C and three license plates 2D, 2E and 2F respectively recognized from the first frame 61 and the second frame 62 by a license plate recognition technology. Next, object distances (i.e., the distance between two license plates) and the string similarities between the objects in the different frames are calculated, and overall scores are also calculated based on the object distances and the string similarities. The overall scores are used to determine the same license plate in different frames.

In the beginning of FIG. 7, one of the license plates of the first frame is selected as a target license plate (step S701). Referring to the flowchart shown in FIG. 4, the object distances and the string similarities between the target license plate of the first frame and all of the license plates of the second frame are calculated. An overall score can also be calculated based on the object distance and the string similarity between the target license plate and each of the license plates of the second frame (step S703). After that, it is determined whether or not the calculations are completed (step S705). The steps S701 to S703 are repeated if the calculation of the object distance or the string similarity of any license plate is not completed until the overall scores of all of the license plates are calculated.

Table 1 is an exemplary example showing the overall scores among all of the license plates. For example, the overall scores between the license plate 1A of the first frame and the license plates 2D, 2E and 2F of the second frame are 100, 90 and 10 respectively. The overall scores between the license plate 1B of the first frame and the license plates 2D, 2E and 2F of the second frame are 95, 60 and 15 respectively. The overall scores between the license plate 1C of the first frame and the license plates 2D, 2E and 2F are 20, 35 and 25 respectively.

	TABLE 1
	Second frame
	license plate	license plate	license plate
first frame	2D	2E	2F
license plate 1A	100	90	10
license plate 1B	95	60	15
license plate 1C	20	35	25

After calculations of the overall scores between the pluralities of license plates in the different frames are completed, an integral overall score for every paired license plates is calculated after the license plates in the first frame and the license plates in the second frame are one-by-one paired (step S707). That is, in the process of the present disclosure, not only the license plate having the highest overall score is relied on to determine the same license plate in different frames, but an integral overall score to be calculated by one-by-one pairing the license plates in different frames is taken into consideration for determining the same license plate in the different frames. For example, an overall score between a license plate 1A of the first frame and a license plate 2D of the second frame is 100, an overall score between a license plate 1B of the first frame and a license plate 2E of the second frame is 60, and an overall score between a license plate 1C of the first frame and a license plate 2F of the second frame is 25. An integral overall score of the above pairing license plates in the different frames is 185 by adding 100, 60 and 25. However, the pairing of the license plates may not be the best if the integral overall score of the pairing is not the highest score.

Thus, the integral overall scores are calculated for all paired license plates acquired respectively from the first frame and the second frame. Therefore, multiple integral overall scores are obtained. The paired license plates with the highest integral overall score are determined as a final decision of paired license plates (i.e., the object-pairing result) (step S709). Next, a score threshold is introduced for filtering out the same license plate in different frames (step S711). For example, the score threshold is set as 50. According to the final decision of paired license plates, the license plates in the different frames are determined as different license plates if the overall score thereof is lower than 50. The license plates in the different frames are assigned with identifiers according to the license-plate pairing result (step S713), in which the same license plates in the different frames are assigned with the same identifier; a new license plate is determined and assigned with a new identifier if the license plates appearing in the different fames are determined as different license plates.

One of the methods for recognizing characters of the string of the license plate can refer to FIG. 8, which is a flowchart illustrating a process of recognizing characters of the string of the license plate according to one embodiment of the present disclosure.

According to one of the embodiments of the present disclosure, the object-tracking system can be installed in an electronic device such as a drive recorder. The electronic device can take a video consisting of continuous frames by a photographing module (e.g., the photographing module 31 of FIG. 3) (step S801) and use a license-plate-recognition module (e.g., the license-plate-recognition module 33 of FIG. 3) that is implemented through collaboration of software and hardware to recognize the positions of one or more in the frames (step S803). In an aspect, an intelligent method can be used to identify the object that is in compliance with the features of a license plate. For example, the intelligent method is able to recognize a boundary of a license plate of each of the vehicles in the frames so as to retrieve an image of the license plate. After that, a string-detection technology (e.g., the string-recognition procedure 333 of FIG. 3) is used to recognize the characters of the string of each of the license plates (step S805).

According to one embodiment of recognizing the characters of the license plate of the present disclosure, the computation module of the object-tracking system performs a confidence-level algorithm on the characters that are frame-by-frame recognized of the string of the license plate so as to calculate a confidence level for each of the characters (step S807). The confidence level for each of the characters relates to a possibility of each of the characters of the license plate being a specific letter or a number. The intelligent model can be a character-recognition model used to recognize the letters and number that compose a license plate number of the license plate. The intelligent model is used for recognizing the characters of the license plate. The system sets a possibility threshold or a confidence level threshold that is referred to for obtaining a recognition result of characters if the recognition result has a maximum possibility among the possibilities greater than the possibility threshold.

Next, when the confidence levels of all or a certain number of the recognizable characters of the license plates have been calculated, a confidence score of all of the characters of each of the license plates can be frame-by-frame calculated based on the confidence levels of the characters of the license plates (step S809). The confidence score of each of the license plates is a statistical value of the confidence levels of multiple characters of each of the license plates. For example, the confidence score can be, but not limited to, an average, a sum or an overall score with weight setting of the confidence levels of all of the characters of the license plate. It should be noted that the weight setting for calculating the overall score gives a higher weight for the character with a higher confidence level.

The processes described in step S807 and step S809 can be used to calculate the confidence score of each of the license plates in the continuous frames. When multiple times of confidence scores are calculated within a period of time (step S811), a highest confidence score can be obtained by continuously calculating multiple confidence scores within the period of time (step S813). Alternatively, multiple confidence scores of multiple license plates extracted from the continuous frames can be calculated at intervals, a frame image having the license plate with the highest confidence score can be obtained before the license plate disappears. In the meantime, when the same license plate appears in multiple continuous frames, the confidence score of each of the multiple continuous frames is calculated, and the frame with the highest confidence score can be obtained. The recognition result of the license plate in the frame with the highest confidence score is regarded as the recognition result of the system (step S815). Recognition of string of the license plate has been completed.

In conclusion, according to the above embodiments relating to the method for tracking object in dynamic scene and the system, one of the objectives of the method is to able to conduct object-tracking in various circumstances, especially the dynamic scene. Especially, the object distance and the string similarity between every two objects in different frames of the continuous frames are taken into consideration for calculating an overall score for determining the same object in the continuous frames. The overall score is referred to for conducting object tracking. Therefore, the method has solved the problem that the conventional technology cannot effectively track the license plate in the dynamic scene.

The foregoing description of the exemplary embodiments of the disclosure has been presented only for the purposes of illustration and description and is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Many modifications and variations are possible in light of the above teaching.

The embodiments were chosen and described in order to explain the principles of the disclosure and their practical application so as to enable others skilled in the art to utilize the disclosure and various embodiments and with various modifications as are suited to the particular use contemplated. Alternative embodiments will become apparent to those skilled in the art to which the present disclosure pertains without departing from its spirit and scope.

Claims

1. A method for tracking an object in a dynamic scene, comprising: capturing a video and obtaining continuous frames;frame-by-frame recognizing one or more objects in each of the frames and choosing a target object in a first frame of the frames;individually calculating one or more distances between the target object in the first frame and one or more objects recognized in a second frame of the frames so as to obtain one or more object distances;recognizing strings of the target object in the first frame and the one or more objects recognized in the second frame;calculating a string similarity between the string of the target object in the first frame and the string of each of the one or more objects recognized in the second frame;calculating an overall score according to the object distance and the string similarity between the target object in the first frame and each of the one or more objects recognized in the second frame; anddetermining whether or not the target object in the first frame is any object appearing in the second frame according to the overall score.

2. The method according to claim 1, wherein the object distance is a distance between a geometric center of the target object in the first frame and the geometric center of each of the one or more objects recognized in the second frame.

3. The method according to claim 2, wherein the string of the target object in the first frame and the string of each of the one or more objects recognized in the second frame are license-plate numbers, each of which is a combination of at least one of multiple English alphabetic letters or multiple numbers; and the string similarity between the strings of two objects is an edit distance.

4. The method according to claim 3, wherein the edit distance is a Levenshtein distance that is a minimum number of single-character edits that is necessary to convert one string into another string between the strings of two objects.

5. The method according to claim 1, wherein the one or more objects to be frame-by-frame recognized are one or more license plates and the one or more license plates are one-by-one numbered.

6. The method according to claim 1, wherein, when the overall score between the target object in the first frame and each of the one or more objects recognized in the second frame is obtained, the overall score is compared with a score threshold so as to determine whether or not the target object in the first frame appears as any object of the one or more objects recognized in the second frame according to a comparison result.

7. The method according to claim 6, wherein, when it is determined that the target object appears in multiple continuous frames of the frames based on the overall score, a unique identifier is assigned to the target object in the continuous frames for tracking the target object.

8. The method according to claim 6, wherein, the object distance between the target object in the first frame and each of the one or more objects recognized in the second frame is assigned with a distance score, the string similarity between the string of the target object in the first frame and the string of each of the one or more objects in the second frame is assigned with a string-similarity score, and the distance score and the string-similarity score are used to calculate the overall score.

9. The method according to claim 8, wherein the overall score is an average of the distance score and the string-similarity score, or the overall score is calculated according to a ratio setting between the distance score and the string-similarity score.

10. The method according to claim 9, wherein an object having the overall score larger than or equal to the score threshold is regarded as the target object in the first frame and assigned with a same identifier with the target object.

11. The method according to claim 9, wherein, where both the first frame and the second frame include multiple objects, the multiple objects of the first frame are one-by-one selected as the target object, the overall score between every target object selected from the first frame and each of the multiple objects recognized in the second frame is calculated; afterwards, multiple integral overall scores are obtained by summing up the overall scores to be calculated by one-by-one pairing the multiple objects of the first frame and the multiple objects of the second frame, and the paired objects having a highest integral overall score are determined to be a same object in different frames by comparing with the score threshold; and the objects in different frames regarded as the same object are assigned with a same identifier.

12. An object-tracking system in a dynamic scene, operated in an electronic device, comprising: a microcontroller;an image processor, electrically connected with the microcontroller, used to perform object recognition and object positioning;a photographing module, electrically connected with the image processor, used to capture a video;wherein the microcontroller performs an object-tracking method comprising: using the photographing module to capture the video and extracting continuous frames including a first frame and a second frame;using the image processor to frame-by-frame recognize one or more objects in the continuous frames and choosing a target object in the first frame;calculating a distance between the target object in the first frame and each of one or more objects recognized in the second frame so as to obtain one or more object distances;recognizing strings of the target object in the first frame and the one or more objects recognized in the second frame;calculating a string similarity between the string of the target object in the first frame and the string of each of the one or more objects recognized in the second frame;calculating an overall score according to the object distance and the string similarity between the target object in the first frame and each of the one or more objects recognized in the second frame; anddetermining whether or not the target object in the first frame is any object appearing in the second frame according to the overall score.

13. The object-tracking system according to claim 12, wherein the object distance is a distance between a geometric center of the target object in the first frame and the geometric center of each of the one or more objects recognized in the second frame.

14. The object-tracking system according to claim 12, wherein the string of the target object in the first frame and the string of each of the one or more objects recognized in the second frame are license-plate numbers, each of which is a combination of at least one of multiple English alphabetic letters or multiple numbers; and the string similarity between the strings of two objects is an edit distance.

15. The object-tracking system according to claim 12, wherein the one or more objects to be frame-by-frame recognized are one or more license plates and the one or more license plates are one-by-one numbered.

16. The object-tracking system according to claim 15, wherein a license plate number of the one or more license plates is frame-by-frame recognized and displayed on a display of the electronic device.

17. The object-tracking system according to claim 12, wherein, in the object-tracking method, when the overall score between the target object in the first frame and each of the one or more objects recognized in the second frame is obtained, the overall score is compared with a score threshold so as to determine whether or not the target object in the first frame appears as any object of the one or more objects recognized in the second frame according to a comparison result; wherein, when it is determined that the target object appears in multiple continuous frames based on the overall score, a unique identifier is assigned to the target object in the continuous frames for tracking the target object.

18. The object-tracking system according to claim 17, wherein the object distance between the target object in the first frame and each of the one or more objects recognized in the second frame is assigned with a distance score, the string similarity between the string of the target object in the first frame and the string of each of the one or more objects in the second frame is assigned with a string-similarity score, and the distance score and the string-similarity score are used to calculate the overall score.

19. The object-tracking system according to claim 18, wherein the overall score is an average of the distance score and the string-similarity score, or the overall score is calculated according to a ratio setting between the distance score and the string-similarity score.

20. The object-tracking system according to claim 19, wherein the object having the overall score larger than or equal to the score threshold is regarded as the target object in the first frame and assigned with a same identifier with the target object; where both the first frame and the second frame include multiple objects, the multiple objects of the first frame are one-by-one selected as the target object, the overall score between every target object selected from the first frame and each of the multiple objects recognized in the second frame is calculated; afterwards, multiple integral overall scores are obtained by summing up the overall scores to be calculated by one-by-one pairing the multiple objects of the first frame and the multiple objects of the second frame, and the paired objects having a highest integral overall score are determined to be a same object in different frames by comparing with the score threshold; and the objects in different frames regarded as the same object are assigned with a same identifier.