The invention relates generally to locomotive navigation, and, in particular, to a system and method for determining which track a locomotive is on when the locomotive is on one of several tracks.
Locomotive video systems are known for their use in rail traffic control. One known locomotive video system employs a signal locating system and a rail navigation system to determine the position that the locomotive vehicle occupies on the railway track, and provides the signal locating system with data as to the whereabouts of the upcoming wayside signal device relative to the position of the vehicle, for example, to guide locomotive vehicles safely and quickly along signaled routes.
Locomotive audio/video recording systems are also known in the art. An exemplary locomotive audio/video recording system is the RailView™ system available from Transportation Technology Group. In such audio/video recording systems, video data and optionally audio data are stored to a high capacity memory device such as a floppy disk drive, hard disk drive or magnetic tape.
Known automatic locomotive navigation systems need to accurately determine a position of a locomotive vehicle for purposes of routing and speed control. Such known locomotive navigation systems, while capable of reliably determining where along a route a locomotive is located when using GPS devices, are still not accurate enough to indicate which track the locomotive is using when there are multiple tracks close to one another.
Accordingly, there exists a need for a reliable system and method for providing locomotive navigation and track identification. It would be both advantageous and beneficial if the system and method could employ video camera equipment and devices already present on the locomotive to detect individual track rails and tracks with or without using a database of prior images of the appearance of the tracks. It would be further advantageous if the system and method were less vulnerable to intermittent failure than known systems and methods that employ, for example, accelerometers that are used to measure rotation of a locomotive as it progresses through switches.
Briefly, in accordance with one embodiment of the present invention, a method is provided for locomotive navigation and track identification. The method, in one embodiment, comprises:
acquiring at least one current video frame via at least one video camera mounted on a locomotive;
processing the at least one current video frame to identify each rail or pairs of rails occupied by the locomotive; and
transmitting information about the identified rail or pairs of rails to a navigation system to determine desired control parameters for the locomotive.
According to another embodiment, a method for locomotive navigation and control comprises:
determining a locomotive track location based on acquired video frame information; and
transmitting the track location to a navigation system to determine desired control parameters for the locomotive based on the track location.
According to yet another embodiment, a video processing system for locomotive navigation and identification comprises:
at least one video camera mounted on a locomotive and configured to acquire at least one video frame; and
a data processing system on-board the locomotive and configured to determine at least one track location based on information extracted from the at least one acquired video frame.
These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:
While the above-identified drawing figures set forth alternative embodiments, other embodiments of the present invention are also contemplated, as noted in the discussion. In all cases, this disclosure presents illustrated embodiments of the present invention by way of representation and not limitation. Numerous other modifications and embodiments can be devised by those skilled in the art which fall within the scope and spirit of the principles of this invention.
The present inventors recognized that knowledge of substantially parallel lines in the world coupled with the location of the principal point can be used to limit the search for railroad tracks within captured images. An introductory discussion is first presented below to provide a better understanding of the embodiments described below with reference to the figures.
A single-dimensional (1D) homography, for example, can be computed between three or more railroad tracks and the actual railroad tracks in the world. This ID mapping provides a direct correspondence between real world and image lines. Thus, putative railroad track locations can be projected into images of the tracks. Image support can then be used to verify the presence/absence of various track configurations.
The location of the foregoing principal point can be determined using various methods. One exemplary method is the intersection of two or more parallel world lines, e.g. the imaged railroad tracks. Another exemplary method is to use the focus of expansion of a moving camera. In a railroad setting, a camera mounted inside of the locomotive can provide the necessary time-series data. Optic flow, point tracking, or other suitable methods can then be used to determine the location of the principal point.
The image to world mapping (the 1D homography) can be computed by manually delineating 3 or more parallel world lines and intersecting the lines with a fourth, non-parallel line. Alternatively, automatic rail detection methods can be used to find the desired lines and then virtually intersect the rails with a fourth line.
Various methods can be used to determine whether or not sufficient image support exists to confirm the presence of a rail or track. Gradient-based and ridge-based methods are two such suitable methods.
The use of geometrical constraints imposed by a world to image mapping has been presented above for use in a railroad setting to provide a background suitable to a better understanding of the embodiments described below with reference to the figures. It can be appreciated that such methods are equally suitable for other “line detection” type problems, such as finding lane or road markings on roads.
The following description presents a system and method for locomotive navigation and track identification using video information, according to particular embodiments. The system and method use a video camera mounted on a locomotive or train to generate video frames as an input to a track identifier, to determine which track a locomotive is on when the locomotive is on one of several nearby tracks. A master navigation system calls upon the track identifier as needed, or the track identifier may be used at regular intervals.
Turning now to the drawings,
Subsequent to acquiring a single video frame or multiple continuous video frames over a desired period of time, frames optionally can be downsampled using conventional image processing techniques, as represented in block 12.
The near-field track vanishing point can be determined from known camera calibration information associated with the single video frame; or the near-field track vanishing point can be computed automatically by processing the video frame(s) data via a CPU, microprocessor, DSP, or other suitable data processing means. This step is represented in block 14 of
The near-field track vanishing point, as used herein, means that point where tracks appear to converge into a single point in the image space when looking in the direction of locomotive travel down the path of the tracks.
The camera calibration information, in one embodiment, is associated with one or more video cameras permanently on-board the locomotive. Using the on-board video camera(s) (lococam), allows the use of already known video camera operating and calibration parameters such as mounting angle and viewing angles, among others, since the video camera is in a permanent fixed position on-board the locomotive. An acquired video image can then be processed using a reverse computational process based on the lococam operating and calibration parameters to identify the three-dimensional position of an object within the image and to determine the near-field track vanishing point.
Downsampling the acquired video image information, such as represented in block 12, is useful when faster processing is desired to gain faster results. Such downsampling allows the use of more powerful processors that are not part of the lococam system, to process the acquired video image information in real time. Faster processing is generally more desirable when processing multiple images because the number of computations required by the computational process increases in a linear relationship with the number of image pixels in the acquired video images.
The vanishing point of the tracks is tracked continuously over a desired period of time as represented in block 16.
Constrained by the track vanishing point, a search is performed to determine each rail or pairs of rails that are occupied by the locomotive, as represented in block 18. Each rail, in one embodiment, is identified using the near-field track vanishing point in a two-dimensional image space. The near-field track vanishing point is that point where all tracks converge in the two-dimensional image space. Angular data associated with each track or pairs of tracks are then used in association with the near-field track vanishing point to identify each track or pairs of tracks.
The foregoing process is then employed to also identify tracks on either side of the occupied track, as represented in block 20. The track identifier can also use information about the layout of the track, which may serve as a geometric constraint to search for tracks, if such information is available. The on-board locomotive system knows its approximate location from GPS measurements or other input data. Based on this knowledge and a track database, the on-board locomotive system may know the number of tracks, the gauge of the tracks, the distances between the tracks and the relative heights of the tracks, among other things. Further, the system may know whether the neighboring tracks are actually visible, and other distinguishing features of those tracks such as ballast material that may aid in their detection.
On-board track identification system 120 includes a track identifier unit 104 that may include without limitation, a computer or processor, logic, memory, storage, registers, timing, interrupts, and the input/output signal interfaces as required to perform the track identifier processing described herein before. The track identifier unit 104, according to one embodiment, receives inputs from a data storage unit 106 that may store a database of track parameters such as described above, at least one on-board video camera (lococam) 102, and a master navigation system 110 via a wireless communication system 130. It will be appreciated that while in an exemplary embodiment, all or most processing is described as resident in the track identifier unit 104, such a configuration is illustrative only. Various processing and functionality may be distributed among one or more system elements without deviating from the scope and breadth of the claims.
The data storage unit 106 is configured with sufficient capacity to capture and record data to facilitate performance of the track identification functions disclosed herein. In one embodiment, data storage unit 106 uses flash memory. Data storage unit 106 may also include non-volatile random access memory (RAM). The data storage unit 106 is comprised in one embodiment, of a solid-state, non-volatile memory of sufficient storage capacity to provide long-term data storage of captured video image data and information, such as but not limited to, video camera calibration information. Once again, it will be appreciated that while the data storage unit 106 is described as a separate entity from the track identification unit 104, either or both could be configured to be separate or combined, as well as being combined with other elements of the on-board system 120. Further, it should be appreciated that while particular partitioning of the processing and functionality is disclosed herein, such partitioning is illustrative only to facilitate disclosure. Many other arrangements and partitions of like functionality will be readily apparent.
The video camera 102, in one embodiment, features aiming and zooming mechanisms that can be externally controlled to aim the camera at an upcoming object with high clarity, even at relatively long distances. Video camera 102 can optionally control lighting effects, resolution, frequency of imaging, data storage, and information concerning video system parameters. Video camera 102 may further take advantage of video technologies that facilitate low/no light image collection or collection of specific images. Examples include infrared and detection of specific images.
One or more video cameras 102 can be employed to acquire the desired track images. The video camera(s) 102 may be directed out the front of the locomotive, to either side, or to the rear of the locomotive; or multiple cameras may be used to capture images from multiple areas.
On-board track identification system 120 also includes, in one embodiment, a communication system 108 that may facilitate a particular type of communication scheme or environment including, but not limited to wireless satellite communications, cellular communications, radio, private networks, a Wireless Local Area Network (WLAN), and the like, as well as combinations including at least one of the foregoing.
A GPS receiver on-board the locomotive in one embodiment, is accurate enough to identify a curve on which the locomotive is located. GPS information may further be coupled with other navigational aids to further facilitate accurate position location and determination. The GPS information may further be coupled with stored information about the track to further facilitate a determination of where the locomotive (and thereby the train) is on the track relative to fixed waypoints or entities.
If any neighboring tracks are occupied, the on-board track identification system 120 may not be able to determine which track the locomotive is on, depending on the arrangement of tracks. When this condition occurs, the track identification system in one embodiment, will report which tracks are occupied, and whether it is able to identify the current track occupied by the locomotive.
Further, unforeseen circumstances may exist that cause the track identifier to fail. When this happens, the track identifier in one embodiment, reports that it cannot determine the current track occupied by the locomotive. The track identifier can be configured to attempt track identification at a later time; or it can be requested to check again later by the master navigation system 110. A fail-safe system such as manual intervention, can also be employed to start the track identification process.
In summary explanation, an automatic locomotive navigation system needs to accurately determine its location for purposes of routing and speed control. While GPS navigation can reliably determine where along a route a locomotive is located, GPS is not accurate enough to tell which track the locomotive is using when there are multiple tracks close to each other. At least one video camera 102 mounted on a locomotive or train is used as an input to a track identifier unit 104 to determine which track a locomotive is on when the locomotive is on one of several tracks. A master navigation system 110 calls upon the track identifier as needed, or the track identifier may be used at regular intervals for routing and speed control purposes, among other things.
The track identifier can optionally be used as in input to a trip optimizer autopilot onboard the locomotive, allowing the autopilot controls to adjust locomotive speed based on speed limits and optimize fuel consumption. A feature of the foregoing locomotive navigation system includes its ability to function in diverse weather, environmental and lighting conditions due to its robust architecture.
Moving now to
Using the weather and light condition constraints from block 602, the locomotive is segmented from an acquired video image as represented in block 604, and such as depicted in
Following the image segmentation described in block 604, the remaining segmented image is then pre-processed to enhance the tracks as represented in block 606.
Once the tracks have been enhanced in the acquired image, desired scene constraints such as but not limited to vanishing point constraints are then used to search for and identify the tracks as represented in block 608. It can be appreciated that although particular embodiments have been described with reference to vanishing point constraints, the invention is not so limited. Any number of other suitable techniques, processes, procedures, methods and algorithms can be employed to implement locomotive navigation and track identification using video in accordance with the principles described herein with or without the use of vanishing point constraints.
Image support is also employed to identify the number and location of the tracks as represented in block 610. The image support may include without limitation, location information from GPS measurements or other input data. Based on this knowledge and a track database, the on-board locomotive system may know the number of tracks, the gauge of the tracks, the distances between the tracks and the relative heights of the tracks, among other things. Further, the system may know whether the neighboring tracks are actually visible, and other distinguishing features of those tracks such as ballast material that may aid in their detection.
The track identification information is then returned to a desired location such as a trip optimizer autopilot or a master navigation system for further processing to determine desired system parameters including without limitation, speed limits as represented in block 611.
A determination is then made as to whether the acquired track image is dark as represented in decision block 804. If the acquired track image is dark, then the acquired image is inverted as represented in block 806, and the full pre-processing continues as represented in blocks 808-814 that represent background substraction, contrast enhancement, edge detection, and orientation filtering steps respectively. If the acquired track image is not dark, then the acquired image is subjected to less pre-processing via bypassing background subtraction 808 and contrast enhancing 810 steps.
Moving now to
The foregoing method of track detection 600 can be employed as well to search over line-pairs instead of individual lines.
The top portion of
While only certain features of the invention have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.
This application claims priority under 35 U.S.C. §119(e)(1) of provisional application Ser. No. 60/963,069, filed Aug. 1, 2007, by Ting Yu et al.
Number | Date | Country | |
---|---|---|---|
60963069 | Aug 2007 | US |