BACKGROUND OF THE INVENTION
Field of the Invention
The present invention relates to a control system of traffic lights and a method thereof, in particular, to a control system using artificial intelligence to control traffic lights and a method thereof.
Description of the Prior Art
Most of the intersections will be set up traffic lights to control vehicles to move or stop. The red light time and the green light time of the traditional traffic lights are set based on experience, and the red light time and the green light time are not adjusted in real time according to the traffic flow. Therefore, when the traffic flow increases, the length of the green light time may be insufficient, such that the traffic flow can't be effectively relieved, which resulting in traffic congestion. In order to solve the problem of traffic congestion, the traditional method is to send the traffic police command on the spot, or let the surveillance personnel remotely control the traffic lights according to the images taken by the cameras installed at the intersection.
However, to control the traffic lights by manpower requires a very large number of personnel, and it is also difficult for personnel to conduct long-term uninterrupted command or monitoring.
SUMMARY OF THE INVENTION
One of objectives of the present invention is to provide a control system of traffic lights and a method thereof.
One of objectives of the present invention is to provide a control system capable of controlling a green light in real time according to the traffic flow and a method thereof.
According to the present invention, a method for determining a time length for developing a timing plan of traffic light control, comprising the steps of: A. acquiring a first image at a preset time point before a red light of a traffic light is turned off, wherein the first image includes vehicles stopped at the red light; B. preprocessing the first image to generate a second image; C. extracting features from the second image by an artificial intelligence algorithm to generate a feature information, and determining a position relationship between vehicles on multiple lanes in the first image based on the feature information; and D. determining the time length by the artificial intelligence algorithm according to the determined position relationship.
According to the present invention, a system for determining a time length for developing a timing plan of traffic light control includes a camera and an artificial intelligence device. The camera captures a first image at a preset time point before a red light of a traffic light is turned off. The first image includes vehicles stopped at the red light. The artificial intelligence device is connected to the camera in a wired or wireless manner. The artificial intelligence device includes an image processing circuit and an artificial intelligence algorithm. The image processing circuit is connected to the camera for preprocessing the first image to generate a second image. The artificial intelligence algorithm configured to perform a set of steps comprises: A. extracting features from the second image to generate a feature information, and determining a position relationship between vehicles on multiple lanes in the first image based on the feature information; and, B. determining the time length according to the determined position relationship.
The control system and method of the present invention can adjust the time of the green light in real time according to the current traffic flow. Therefore, it may effectively relieve the traffic flow and solve the problem of traffic congestion.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 shows a schematic diagram of an application of a control system of the present invention.
FIG. 2 shows an embodiment of image processing in the present invention.
FIG. 3 shows an embodiment of a control method of the present invention.
FIG. 4 shows a first embodiment of an artificial intelligence device in FIG. 1.
FIG. 5 shows operation steps of an artificial intelligence device in FIG. 4.
FIG. 6 shows a second embodiment of an artificial intelligence device in FIG. 1.
FIG. 7 shows a first operation embodiment of an artificial intelligence device in FIG. 6.
FIG. 8 shows a second operation embodiment of an artificial intelligence device in FIG. 6.
FIG. 9 shows a schematic diagram of the control system of the present invention applied to a road with a plurality of lanes.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
FIG. 1 shows a schematic diagram of the application of the traffic light control of the present invention. The control system 10 of the present invention uses artificial intelligence (AI) to control the traffic lights 12 mounted at an intersection. The traffic lights 12 include a red light 122, a yellow light 124 and a green light 126. In an embodiment, the traffic lights 12 may also include two or more green lights, such as a green light indicating turning left (left-turn signal) or a green light indicating turning right (right-turn signal). With reference to FIG. 1, each component in the control system 10 may be arranged on the same bracket as the traffic lights 12, but it is not limited to this. The control system 10 includes a camera 102 and an artificial intelligence device 104. The camera 102 is used to shoot the intersection to obtain a video of the intersection, and the video is composed of a plurality of continuous images. In an embodiment, the camera 102 is a fisheye camera. A range captured by the camera 102 includes at least a section of a road in front of the traffic lights 12, and the road may include one or more lanes. When the red light 122 of the traffic lights 12 is turned on, various types of vehicles will stop in front of the stop line 14 at the intersection. Then, the camera 102 captures a video of at least one vehicle that is waiting for the red light 122. The vehicles mentioned here include various transportation such as cars and scooters. The camera 102 captures an image A at a preset time point t before the red light 122 is turned off, for example, 3 seconds before the red light 122 is turned off. The camera 102 transmits the image A (as shown in FIG. 2) to the artificial intelligence device 104 in a wired or wireless manner. In another embodiment, the camera 102 may transmit the video to the artificial intelligence device 104, and the artificial intelligence device 104 acquires the image A taken at the preset time point t from a plurality of continuous images of the video. The artificial intelligence device 104 is connected to the traffic lights 12 in a wired or wireless manner. The artificial intelligence device 104 determines a time length T according to the image A, and the time length T is used to control the time for the turning on the green light 126. The time length T is an estimated value. In the embodiment of FIG. 1, the time length T is estimated by the artificial intelligence device 104 and represents the time that all the vehicles in at least one lane in the image A pass the stop line 14 after the red light 122 is turned off and the green light 126 is turned on. However, the time length T of the present invention is not limited to the time for all vehicles to pass the stop line 14. In an embodiment, the artificial intelligence device 104 controls the green light 126 according to the time length T. In another embodiment, the artificial intelligence control device 104 transmits the time length T to a remote server, and the server generates a timing plan according to the time length T to control the green light 126.
It can be understood from the above description that the present invention provides a control method for traffic lights as shown in FIG. 3. With reference to FIGS. 1 and 3, the control method of the present invention is to acquire an image A at a preset time point t before the red light 122 is turned off, as shown in the step S10. The image A includes at least one vehicle waiting for the red light 122. In an embodiment, the camera 102 may be used to capture at the preset time point t to acquire the image A, and the camera 102 then transmits the image A to the artificial intelligence device 104 in a wired or wireless manner. In another embodiment, the camera 102 shoots a plurality of continuous images to generate a video, the camera 102 transmits the video to the artificial intelligence device 104, and then the artificial intelligence device 104 acquires the image A taken at the preset time point t from the plurality of continuous images of the video. The step S12 is performed after image A is acquired. The step S12 comprises determining a time length T according to image A. The time length T is generated through an artificial intelligence algorithm in the artificial intelligence device 104. In an embodiment, the time length T is estimated by the artificial intelligence device 104 and represents the time that all the vehicles in at least one lane in the image A pass the stop line 14 after the red light 122 is turned off and the green light 126 is turned on. Finally, the present invention controls the green light 126 according to the time length T, as shown in the step S14. In an embodiment, the artificial intelligence device 104 controls the green light 126 according to the time length T. In another embodiment, the artificial intelligence control device 104 transmits the time length T to a remote server, and the server generates a timing plan according to the time length T to control the green light 126.
FIG. 4 shows a first embodiment of the artificial intelligence device 104 of FIG. 1. The artificial intelligence device 104 includes an artificial intelligence (AI) algorithm 1042 and a controller 1044, wherein the AI algorithm 1042 may be implemented by hardware circuit or software. FIG. 5 shows an operation embodiment of the artificial intelligence device 104 in FIG. 4. The steps S121 and S141 in FIG. 5 correspond to the steps S12 and S14 in FIG. 3, respectively. With reference to FIGS. 4 and 5, after the artificial intelligence device 104 acquires the image A, the AI algorithm 1042 in the artificial intelligence device 104 generates the time length T according to the image A, as shown in step S121. The AI algorithm 1042 may be, but is not limited to, a Convolutional Neural Network (CNN). CNN includes a feature extraction part and a regression part. The feature extraction part is used to extract features from the image to generate feature information, and the regression part generates a time length T according to the feature information. The controller 1044 controls the time length for turning on the green light 126. In the embodiment of FIG. 4, the controller 1044 receives the time length T, and generates a control signal Sc to the traffic lights 12 according to the time length T to control the time length for turning on the green light 126, as shown in step S141 of FIG. 5.
FIG. 6 shows a second embodiment of the artificial intelligence device 104 in FIG. 1. In addition to the AI algorithm 1042 and the controller 1044, the artificial intelligence device 104 in FIG. 6 further includes an image processing circuit 1046 between the camera 102 and the AI algorithm 1042. FIG. 7 shows a first operation embodiment of an artificial intelligence device 104 in FIG. 6. A combination of steps S122 and S123 in FIG. 7 can be understood as an embodiment of the step S12 in FIG. 3, and the step S141 in FIG. 7 corresponds to the step S14 in FIG. 3. With reference to FIGS. 6 and 7, after the artificial intelligence device 104 acquires the image A, step S122 is performed first. In the step S122, the image processing circuit 1046 preprocesses the image A. The preprocessing includes classifying and labeling all the vehicles in the image A to generate a first labeled image B (as shown in FIG. 2). The image processing circuit 1046 may use, but is not limited to, a computer vision algorithm or an AI vision detection algorithm to classify and label at least one vehicle in the image A. In an embodiment, the way to classify vehicles includes classification by vehicle types, such as classifying into buses, cars, scooters, and trucks. In an embodiment, the vehicles in image A may be labeled with different colored labeling frames. For example, as shown in FIG. 2, cars are labeled with a green labeling frame 16, scooters are labeled with a blue labeling frame 18, and buses and trucks are labeled with a red labeling frame 20. Then, step S123 is performed to analyze the first labeled image B by the AI algorithm 1042 to generate the time length T. Different types of vehicles have different reaction times from when the green light 126 is turned on to when they start to move. Generally, the reaction time of scooters is faster, and the reaction time of buses and trucks is slower. Therefore, vehicles behind buses or trucks may take more time to pass stop line 14 or the intersection. The AI algorithm 1042 of the present invention estimates the time length T for all vehicles to pass the stop line 14 based on the position relationship between different types of vehicles in the labeled image B. Finally, the controller 1044 generates a control signal Sc to the traffic lights 12 according to the time length T to control the time length for turning on the green light 126, as shown in the step S141.
FIG. 8 shows a second operation embodiment of the artificial intelligence device 104 of FIG. 6. A combination of steps S122, S124 and S125 in FIG. 8 can be understood as another embodiment of step S12 in FIG. 3, and the step S141 in FIG. 8 corresponds to the step S14 in FIG. 3. With reference to FIGS. 6 and 8, after the artificial intelligence device 104 acquires the image A, the image processing device 1046 preprocesses the image A, as shown in steps S121 and S124. Specifically, the image processing device 1046 first classifies and labels all the vehicles in the image A to generate a first labeled image B, as shown in the step S122. Then, in order to prevent objects (such as road signs 22, street lights 24, road trees 26, and buildings 28 shown in FIG. 1) other than vehicles from affecting the following determination, the image processing circuit 1046 ignores the objects other than the vehicle in the first labeled image B to generate a second labeled image C, as shown in step S124. In the embodiment of FIG. 2, the second labeled image C has a black background and uses different color blocks to indicate different types of vehicles. For example, cars are labeled with a green color block 30, scooters are labeled with a blue color block 32, and buses and trucks are labeled with a red color block 34. The second labeled image C of the present invention is not limited to the embodiment shown in FIG. 2. Subsequently, step S125 is performed to analyze the second labeled image C by the AI algorithm 1042 to generate the time length T. Similarly, the AI algorithm 1042 of the present invention estimates a time length T according to the position relationship between different types of vehicles in the second labeled image C. Finally, the controller 1044 generates a control signal Sc to the traffic lights 12 according to the time length T to control the time length for turning on the green light 126, as shown in the step S141.
The AI algorithm 1042 of the artificial intelligence device 104 in FIGS. 4 and 6 has to be trained in advance to have the ability to estimate a time length T. The training method includes preparing many different images I. These images I may be taken at the intersection as shown in FIG. 1, for example. These images I are taken at different time points, and each image I is taken at a preset time point t before the red light 122 is turned off. In another aspect, an actual time Tr for all vehicles in each image I to pass through the stop line 14 has to be measured. These images I and their respective actual times Tr are provided to a training program Pt. The training program Pt has the same architecture as the AI algorithm 1042, such as the CNN architecture. The training program Pt obtains a set of coefficients for estimating the time length T according to the images I and the corresponding actual times Tr. The AI algorithm 1042 uses the set of coefficients to perform operations and analyze the image A, and estimate the time length T for all vehicles in the image A to pass the stop line 14. The above training process may be understood as making the training program Pt learn to estimate how much time is needed for the vehicle in the image to pass the stop line 14 according to the image. Through the above training process, the AI algorithm 1042 has the ability to estimate a time length T according to the image A.
In different embodiments, the traffic lights 12 may further include a right turn light 126″ and a left turn light 126′; in addition to the through lane 40, the road where the traffic light 12 is located further includes a left turn lane 42 and a right turn lane 44, as shown in FIG. 9. According to the present invention, the image A captured by the camera 12 may be divided into an image Ad of the through lane 40, an image Al of the left turn lane 42, and an image Ar of the right turn lane 44. In other words, the present may obtain the image Ad of the through lane, the image Al of the left turn lane, and/or the image Ar of the right turn lane from the image A. In an embodiment, the artificial intelligence device 104 recognizes different lanes by detecting the arrows 402, 422, and 442 on the road in the image A. In general, the left turn lane 42 will be labeled with the arrow 422 representing a left turn, the right turn lane 44 will be labeled with the arrow 442 representing a right turn, and the through lane 40 will be labeled with the arrow 402 representing going straight. The AI algorithm 1042 generates the time length T1 according to of the image Ad to control the green light 126, generates the time length T2 according to the image Al to control the left turn light 126′, and generates the time length T3 according to the image Ar to control the right turn light 126″. In view of this situation, to train the AI algorithm 1042 to estimate the time T1, it is necessary to provide many images of the through lane to the training program Pt. To train the AI algorithm 1042 to estimate the time T2, it is necessary to provide many images of the left turn lane to the training program Pt. To train the AI algorithm 1042 to estimate the time T3, it is necessary to provide many images of the right turn lane to the training program Pt. The rest of the details are similar to the aforementioned training process, and will not be repeated here.
In the embodiment of FIGS. 4 and 6, the controller 1044 is set in the artificial intelligence device 104, and the controller 1044 may be a computer or a hardware circuit for controlling the traffic lights 12. In different embodiments, the controller 1044 may be arranged outside the control system 10, for example, the controller 1044 may be in a remote traffic control center (not shown). In this case, the control system 10 transmits the time length T to the controller 1044 in the remote control center in a wired or wireless manner, and then the controller 1044 generates a control signal Sc according to the time length T to control the lighting time of the green light 126 via a wired or wireless network.
While the present invention has been described in conjunction with preferred embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and scope thereof as set forth in the appended claims.
Patent Application
20230377332
