The present disclosure relates to a field of computer technology, and particularly to a method and an apparatus for fusing traffic markings, a storage medium, and an electronic device.
In a crowd-sourced fusion scenario of high-precision data, when traffic marking data is collected (for example, linear data is collected by using environmental perception technologies such as an image perception technology and a laser perception technology), since data is collected frame by frame, data collected between the frames may not be completely fitted, resulting in a misalignment phenomenon after the linear data of different frames is combined. At present, misaligned linear data is generally fused in a linear fusion method. In the linear fusion method, it is necessary to calculate the homonymous point pairs and adopt a square difference method, which requires a large amount of calculation.
A purpose of the present disclosure is to provide a method and apparatus for fusing traffic markings, a storage medium and an electronic device, to improve a fusion efficiency of traffic markings.
According to a first aspect of embodiment of the present disclosure, a method for fusing traffic markings is provided. The method includes:
According to a second aspect of the embodiment of the present disclosure, a computer readable storage medium is provided, having a computer program stored thereon, when the program is executed by a processor, the steps of the method according to the first aspect of the present disclosure are implemented.
According to a third aspect of the embodiment of the present disclosure, an electronic device is provided, including:
The accompanying drawings, which constitute a part of this specification, are provided for better understanding of the present disclosure, and serve to explain the present disclosure together with the following embodiments, which shall not be construed to limit the present disclosure.
Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely intended to illustrate and explain the present disclosure, but shall not be construed to limit the present disclosure.
Traffic markings are also road traffic markings, which refer to identifiers for indicating traffic information such as guidance, restriction, warning and the like to traffic participants by lines, arrows, characters, facade marks, protruding road signs, contour marks and the like on a road surface of a road. Although the traffic markings are stationary, as set forth in the background, in a process of collecting road data, frame-by-frame collection is performed by using an existing data collection mode (for example, an image collection mode and a laser collection mode). Due to a viewing angle difference between frames, there may be a deviation in collections for a same group of the traffic markings. That is to say, a misalignment may occur at the connection of the same group of the traffic markings. Therefore, for this situation, the traffic markings with the misalignment need to be fused. At present, a common fusion mode in the related art is to perform a geometric adjustment based on an adjustment principle. In an adjustment process, multi-source data is classified into an “adjustment graph” and a “reference graph”, and then a coordinate displacement equation is established based on pairs of matching points having a correspondence in the “adjustment graph” and “reference graph”. In order to calculate a geometric adjustment amount, the coordinate displacement equation and a shape equation generally need to be designed, and coordinate values are solved by adopting a constrained least square method. However, the above method requires a large amount of calculation and has a low adjustment efficiency.
In order to solve the above problems, a method and apparatus for fusing the traffic markings, a storage medium, and an electronic device are provided in the present disclosure, so as to improve a fusion efficiency of the traffic markings.
At step 11, a first local part overlapped with a marking set to be fused is obtained from a target marking set, and a second local part overlapped with the target marking set is obtained from the marking set to be fused.
At step 12, based on a target transformation relationship, a first transformed marking set is obtained after transformation of the first local part and a second transformed marking set is obtained after transformation of the second local part.
At step 13, a first translation parameter capable of fusing the second transformed marking set and the first transformed marking set is determined.
At step 14, a second translation parameter is determined by using the first translation parameter based on the target transformation relationship.
At step 15, the marking set to be fused is moved using the second translation parameter.
The method provided in the present disclosure is used for realizing a fusion of traffic markings in the target marking set and the marking set to be fused. Therefore, before the step 11 is executed, the marking set to be fused corresponding to the target marking set needs to be determined for a given target marking set.
The target marking set and the marking set to be fused are both taken from marking sets of road data, and each marking set includes a group of traffic markings. The road data is data related to a road that can be collected in a data collection process, and the road data includes a marking set composed of a group of traffic markings. Here, the group of traffic markings may be considered to be several traffic markings with similar positions, for example, the traffic markings of a same lane, or the traffic markings of a same road, etc. For example, traffic markings may be lane lines. For another example, traffic markings may be crosswalk lines.
As described in the background, a case where a same marking set has a misalignment is usually derived from collections of different frames. Therefore, the two frames for which the collected road data is fused may be specified in the present disclosure. For example, in an image collection mode, two frames of images collected in turn may be used as processing objects of a fusion operation.
In a possible implementation, the marking set to be fused corresponding to the target marking set may be determined by the following steps.
Based on an area occupied by the target marking set, a marking set whose occupied area overlaps with the area occupied by the target marking set is determined from the road data as the marking set to be fused.
In general, it is considered that a misalignment phenomenon only occurs to the traffic markings corresponding to the same road or the same lane, and the misalignment can cause the areas occupied by the markings to be partially overlapped. On the basis of the above, the marking set to be fused corresponding to the target marking set can be identified. That is, based on a position where the target marking set is located, the marking set in the road data that overlaps with the area of the target marking set is determined, and the determined marking set is determined as the marking set to be fused.
Optionally, in some cases, there may be a phenomenon that the number of the traffic markings of the target marking set is inconsistent with the number of the traffic markings of the marking set to be fused. In this case, if a subsequent processing is directly performed, a data processing error may be caused due to inconsistency of the numbers of the traffic markings, or even the fusion fails. Therefore, in a possible implementation, before the step 11 is executed, on the basis of steps shown in
At step 21, a first number of traffic markings in the target marking set and a second number of traffic markings in the marking set to be fused are obtained.
At step 22, if the first number is inconsistent with the second number, a smaller one of the first number and the second number is determined as the target number.
At step 23, a traffic marking is deleted from the target marking set or the marking set to be fused, so that the number of the traffic markings in the target marking set and the number of the traffic markings in the marking set to be fused are both the target number.
In general, if the number of traffic markings in the target marking set is inconsistent with the number of traffic markings in the marking set to be fused, consistency of numbers of traffic markings in the two index line sets can be realized by deleting one or more markings from the set with a larger number or adding one or more target markings to the set with a less number. However, problems of slow data processing speed, accuracy reduction, etc., are easily caused by adding the traffic marking in a processing scenario of the present solution. Therefore, a manner of deleting a traffic marking is usually adopted.
For example, the step 23 may include the following steps:
If the first number is greater than the second number, a specified number of the traffic markings at the target side are deleted from the target marking set. If the second number is greater than the first number, the specified number of the traffic markings at the target side are deleted from the marking set to be fused. The specified number is a difference between the first number and the second number.
For the marking set shown in
In this way, the target marking set and the marking set to be fused are preprocessed, so that the numbers of the traffic markings contained in the two sets are the same, which facilitates improving subsequent data processing efficiency and a processing accuracy.
Referring to
A local marking set, i.e., the first local part, corresponding to an intersection part may be obtained from the target marking set based on an overlapped area of the target marking set and the marking set to be fused. Similarly, the local marking set, i.e., the second local part, corresponding to the intersection part is obtained from the marking set to be fused. In
At the step 12, based on the target transformation relationship, the first transformed marking set is obtained after transformation of the first local part, and the second transformed marking set is obtained after transformation of the second local part.
The target transformation relationship enables the second local part after transformation to be perpendicular to a specified coordinate axis. The specified coordinate axis is a horizontal axis or a vertical axis in a preset plane rectangular coordinate system. The preset plane rectangular coordinate system is constructed according to the earth coordinate system, for example, the horizontal axis and vertical axis are constructed with reference to latitude and longitude.
For example, the target transformation relationship may be determined by an affine transformation. Converting a pattern to a desired position by means of the affine transformation is common general knowledge in the related art, and details are not described herein.
After the target transformation relationship is obtained, the first transformed marking set may be generated by performing a transformation on the first local part based on the target transformation relationship. The second transformed marking set by performing a transformation performed on the second local part based on the target transformation relationship.
The target transformation relationship can make the second local part after transformation perpendicular to the specified coordinate axis (the horizontal axis or the vertical axis in the preset plane rectangular coordinate system). Therefore, after the second local part is transformed based on the target transformation relationship, the second local part can be perpendicular to the specified coordinate axis. In this way, in a subsequent translation process, it is only required to perform a translation in a specified coordinate axis direction, to obtain an optimal solution capable of fusing the two marking set.
At the step 13, the first translation parameter capable of fusing the second transformed marking set with the first transformed marking set is determined.
In a possible implementation, the step 13 may include the following steps:
The first bounding rectangle and the second bounding rectangle are both the minimum bounding rectangles of the patterns. The minimum bounding rectangle is a minimum bounding rectangle surrounding a primitive (abbreviation of a basic graphic element), with two adjacent edges being respectively parallel to the horizontal axis and the vertical axis. Correspondingly, the first bounding rectangle is the minimum bounding rectangle surrounding the first transformed marking set and with two adjacent edges being respectively parallel to the horizontal axis and the vertical axis in the preset plane rectangular coordinate system. The second bounding rectangle is the minimum bounding rectangle surrounding the second transformed marking set and with two adjacent edges being respectively parallel to the horizontal axis and the vertical axis in the preset plane rectangular coordinate system.
Optionally, performing the plurality of translation operations on the second transformed marking set, recording the intersection area of a second bounding rectangle corresponding to the second transformed marking set and the first bounding rectangle in each translation, and taking a translation parameter corresponding to a translation with a maximum intersection area as the first translation parameter may include following first to sixth steps.
In a first step, an initial position of the second transformed marking set without being subject to any translation is recorded, and an intersection area of the second bounding rectangle corresponding to the second transformed marking set at the initial position and the first bounding rectangle is determined as an initial intersection area.
In a second step, the second transformed marking set is translated in a specified direction by a specified distance, so that the second transformed marking set reaches a target position of this translation.
In a third step, the intersection area of the second bounding rectangle corresponding to the second transformed marking set at the target position and the first bounding rectangle is determined as a target intersection area.
In a fourth step, the target intersection area is compared with a reference intersection area.
The reference intersection area is determined in the translation process of translating in the specified direction, which is the intersection area of the second bounding rectangle of the second transformed marking set before being subject to this translation and the first bounding rectangle. The reference intersection area is the initial intersection area under an initial condition.
If the target intersection area is less than the reference intersection area, a fifth step is performed.
In the fifth step, translating the second transformed marking set in the specified direction is stopped.
In the sixth step, after a translation operation on the second transformed marking set is stopped, the translation parameter of an intermediate position relative to the initial position is used as the first translation parameter, the intersection area of the second bounding rectangle corresponding to the second transformed marking set at the intermediate position and the first bounding rectangle is maximum in a historical translation process.
If the target intersection area is larger than the reference intersection area after comparing the target intersection area with the reference intersection area in the above fourth step, the second step is returned. The second step and its subsequent steps are performed again. That is, a translation operation is performed and the intersection area is calculated again, and the intersection area is compared with a new reference intersection area again. It is judged whether it is necessary to recycle the translation process once again based on a comparison result until it is determined that the fifth step can be performed.
The specified direction is a direction parallel to the specified coordinate axis, and the specified direction includes a first direction and a second direction which are opposite to each other.
That is, the second transformed marking set may be translated in the first direction by the specified distance based on the initial position of the second transformed marking set, and the target intersection area corresponding to this translation is calculated, when the second transformed marking set is moved to attempt to find a most appropriate first translation parameter. If the target intersection area corresponding to this translation is increased compared to the target intersection area of a previous translation, the second transformed marking set is continued to be moved in the first direction based on the target position reached by this translation. If the target intersection area corresponding to this translation is reduced compared to the target intersection area of the previous translation, the second transformed marking set is stopped to be translated in the first direction. The translation processing mode in the second direction is the same as that in the first direction, and details are not described herein. It should be noted that, regardless whether the translation is performed in the first direction or in the second direction, the second transformed marking set is located at the initial position in the first step. Translation operations in the first direction and in the second direction are independent of each other and do not affect each other, and the translation operations in the first direction and in the second direction are not limited in a strict sequence.
Until translations in the first direction and the second direction are both stopped, a series of intersection areas in the historical translation process are recorded. A maximum intersection area is selected from the intersection areas. The translation parameter corresponding to the maximum intersection area is used as the first translation parameter.
Optionally, in order to ensure the accuracy of the first translation parameter, the specified distance may be selected as a unit distance. That is, the unit distance may be moved compared to the position before moving each time.
A process of determining the first translation parameter is illustrated below in conjunction with
In the example, a distance change (the specified distance) of each translation is a distance between two adjacent lane lines.
First, the translation process in which the first direction is the left direction is described.
On a basis of
The translation process in which the second direction is the right direction is described below.
On the basis of
That is, on the basis of
After the translations in the left and right directions are ended, the execution condition of the sixth step is satisfied. Therefore, the translation parameter of the intermediate position relative to the initial position is used as the first translation parameter, in which the intersection area of the second bounding rectangle corresponding to the second transformed marking set at the intermediate position and the first bounding rectangle is the maximum intersection area in the historical translation process. In the example, a change parameter of a position of the second transformed marking set in
In the above process, since only a graphic movement in the direction parallel to the specified coordinate axis is involved, a calculation amount during the movement is very small, so that the efficiency of obtaining the first translation parameter can be effectively improved.
At the step 14, the second translation parameter is determined by using the first translation parameter based on the target transformation relationship.
After the first translation parameter is obtained, based on the target transformation relationship, the inverse process of obtaining the target transformation relationship is used to reversibly push out a translation parameter corresponding to the first translation parameter before transformation, that is, the second translation parameter. For example, if the target transformation relationship is obtained by the affine transformation, the second translation parameter can be obtained by an anti-affine transformation based on the first translation parameter. The second translation parameter is the translation parameter referenced by the actual movement of the marking set to be fused.
At the step 15, the marking set to be fused is moved by using the second translation parameter.
After the second translation parameter is obtained, the second translation parameter may be used to move the marking set to be fused, to make the marking set to be fused fuse with the target marking set, therefore a fused marking set is obtained.
According to the technical solution, a first local part overlapped with the marking set to be fused is obtained from the target marking set, a second local part overlapped with the target marking set is obtained from the marking set to be fused. Based on a target transformation relationship, a first transformed marking set is obtained after transformation of the first local part, a second transformed marking set is obtained after transformation of the second local part. A first translation parameter capable of fusing the second transformed marking set with the first transformed marking set is determined. A second translation parameter is derived by using the first translation parameter based on the target transformation relationship. The marking set to be fused is moved by using the second translation parameter. In this way, in a process of fusing the traffic markings, only a simple calculation mode needs to be used, which is beneficial to improve a fusion efficiency and accuracy.
The first obtaining module 41 is configured to obtain a first local part overlapped with a marking set to be fused from a target marking set and a second local part overlapped with the target marking set from the marking set to be fused.
The generating module 42 is configured to, based on a target transformation relationship, generate a first transformed marking set transformed from the first local part and a second transformed marking set transformed from the second local part.
The first determining module 43 is configured to determine a first translation parameter capable of fusing the second transformed marking set with the first transformed marking set.
The derivation module 44 is configured to determine a second translation parameter by using the first translation parameter based on the target transformation relationship.
The moving module 45 is configured to move the marking set to be fused by using the second translation parameter.
Optionally, the apparatus 40 further includes a second determining module.
The second determining module is configured to for a given target marking set, determine the marking set to be fused corresponding to the target marking set. The target marking set and the marking set to be fused are both taken from marking sets of road data, and each marking set includes a group of the traffic markings.
Optionally, the second determining module is configured to determine a marking set whose occupied area overlaps with an area occupied by the target marking set based on the area occupied by the target marking set from the road data as the marking set to be fused.
Optionally, the apparatus 40 further includes a second obtaining module, a third determining module and a deleting module.
The second obtaining module is configured to obtain a first number of the target markings in the target marking set and a second number of the target markings in the marking set to be fused, before obtaining the first local part overlapped with the marking set to be fused from the target marking set and the second local part overlapped with the target marking set from the marking set to be fused.
The third determining module is configured to determine a smaller one of the first number and the second number as a target number, in response the first number being inconsistent with the second number.
The deleting module is configured to delete a traffic marking from the target marking set or the marking set to be fused, to enable a number of the traffic markings in the target marking set and a number of the traffic markings in the marking set to be fused to be the target number.
Optionally, the deleting module includes a first determining submodule, a second determining submodule, a third determining submodule, and a deleting submodule.
The first determining submodule is configured to determine a first distance between a traffic marking at a first side edge of the target marking set and a traffic marking at the first side edge of the marking set to be fused.
The second determining submodule is configured to determine a second distance between a traffic marking at a second side edge of the target marking set and a traffic marking at the second side edge of the marking set to be fused.
The third determining submodule is configured to take one side corresponding to a larger one of the first distance and the second distance as a target side.
The deleting submodule is configured to delete the traffic marking at the target side from the target marking set or the marking set to be fused.
Optionally, the deleting submodule is configured to:
The specified number is a difference between the first number and the second number.
Optionally, the target transformation relationship enables the second local part after transformation to be perpendicular to a specified coordinate axis.
Optionally, the target transformation relationship is obtained through an affine transformation.
Optionally, the first determining module 43 includes a fourth determining submodule and a fifth determining submodule.
The fourth determining submodule is configured to determine a first bounding rectangle corresponding to the first transformed marking set.
The fifth determining submodule is configured to perform a plurality of translation operations on the second transformed marking set, record the intersection area of a second bounding rectangle corresponding to the second transformed marking set and the first bounding rectangle in each translation, and take a translation parameter corresponding to a translation with a maximum intersection area as the first translation parameter.
Optionally, the fifth determining submodule is configured to:
Optionally, the fifth determining submodule is configured to:
Optionally, the specified direction is a direction parallel to the specified coordinate axis, and the specified direction includes a first direction and a second direction are opposite to each other.
Optionally, traffic markings are lane lines.
With regard to the apparatus in the above embodiments, the specific way in which each module performs the operation has been described in detail in the embodiments of the method and will not be elaborated here.
In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950. The power supply component 1926 may be configured to perform power management of the electronic device 1900. The communication component 1950 may be configured to implement communications of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 may further include an input/output (I/O) interface 1958. The apparatus 1900 may operate an operating system stored in the memory 1932, for example, Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.
In another example embodiment, a computer-readable storage medium including program instructions is further provided, and when the program instructions are executed by the processor, the steps of the above-mentioned method for fusing traffic markings are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 1932 including the program instructions, and the program instructions may be executed by the processor 1922 of the electronic device 1900 to execute the above-mentioned method for fusing traffic markings.
In another exemplary embodiment, a computer program product is further provided. The computer program product includes computer programs that may be executed by a programmable apparatus, and the computer program possesses a code part configured to execute the above method for fusing traffic markings when executed by the programmable apparatus.
The embodiments of the present disclosure are described in detail above with reference to the accompanying drawings. However, the present disclosure is not limited to the specific details of the above-mentioned embodiments, and various simple modifications can be made to the embodiments of the present disclosure within the technical concept of the present disclosure, which also belong to the protection scope of the present disclosure.
In addition, it should be noted that various specific technical features described in the above-mentioned embodiments can be combined in any suitable manner if not contradictory. In order to avoid unnecessary repetition, various possible combinations will not be elaborated herein.
In addition, various embodiments of the present disclosure can also be combined arbitrarily, as long as they do not violate the spirit of the present disclosure, and they should also
Number | Date | Country | Kind |
---|---|---|---|
202110369281.8 | Apr 2021 | CN | national |
This application is a U.S. national phase application of International Application No. PCT/CN2022/077532, filed on Feb. 23, 2022, which is based on and claims priority to Chinese patent application NO. 202110369281.8, filed on Apr. 6, 2021, the entire contents of which are incorporated herein by reference.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CN2022/077532 | 2/23/2022 | WO |