Patent Application
20240060245
The present invention relates in general to the field of railroad maintenance. More particularly, the present invention relates to a computer vision-based railway component detector system that detects various components associated with a railway. For instance, the present invention may be used to detect specific components associated with a railway, for instance spike holes, after which a chemical solution can automatically be injected into spike holes of railroad ties during track maintenance. It may also detect the presence of a tie as part of an indexing operation that is a precursor to spike hole filling operation. The present invention could similarly be used to locate ties, tie plates and other features components associated with a railway. The invention also relates to railroad maintenance machines having such a detector system and to a method of using such a system.
Rail anchors, used to secure a rail to railroad ties, typically are held in place by spikes driven into underlying ties that run perpendicular to the rails. These spikes are removed during a variety of maintenance operations, such as a rail re-lay, anchor or tie plate replacement, etc. As a result of the pulling of the spikes that hold the rails to their plates, several holes remain in the tie at the location vacated by the plate. It is usually desirable to fill or “plug” these “spike holes” to prevent rot and water freezing in the open spike holes, which can cause damage to the tie. In addition, should a spike be inserted into an existing spike hole, something of a substance should be in the location to retain the hold-down force of the spike within the tie.
The classic approach to filling spike holes was simply to manually insert cedar plugs into the holes as part of the rail re-lay operation. These plugs initially were inserted by laborers walking along the railway. Later, machines were developed that allowed operators to insert plugs using hand-held tools.
More recently, several different chemical solutions have been developed that are injected into the holes and then react either with a component of the injected material, chemical, or water to form a relatively hard substance that approximates the physical characteristics of wood. Examples of such solutions include a polyurethane-based chemical, an epoxy-based chemical, and a water-based chemical. The first way of injecting these materials was to manually inject the solution into the spike holes using a caulk gun type device or “gun” that simultaneously mixes the constituent chemicals of the solution and injects the solution into the spike holes. This technique is still in use, but generally is limited to relatively small-scale applications such as replacing a short section of railway.
Vehicles have been developed permitting riding operators to manually inject solution into spike holes using guns of the type historically used by walking operators but supplied with chemicals via one or more on-board tanks rather than a self-contained cartridge on the gun. The machine may be either self-propelled and move along the rails or mounted on the back of a pickup truck or the like. These machines typically include a single gun that is manually directed and activated by an operator. Other than being transported by a vehicle and having tanks, these types of devices are, in essence, the same as the traditional caulk gun style operation.
In all of these machines, the guns are controlled, manipulated and triggered by operator rather than being mounted on a work head and operated automatically. In addition, each of these prior machines or techniques required a dedicated operator to each gun rather than permitting a single operator to operate multiple guns.
A number of the drawbacks experienced with previous systems were largely alleviated with the introduction of the improved spike hole filling machine and method described in U.S. Pat. No. 9,617,692 (“the '692 patent”), which is assigned to the assignee of the present application and which is incorporated herein by reference. This machine permits a riding operator to inject a hole filling solution into spike holes as the machine is propelled along the tracks. The injector is provided on a movable workhead so as to be movable multi-axially by the operator using joysticks and other controls to align the injector with the spike holes and inject solution. This machine represents a dramatic improvement over earlier machines in terms of both accuracy and speed.
Nevertheless, further improvements are desired. Hole location and filling operations are still controlled wholly by an operator in machine described in the '692 patent. This requires a skilled operator on the machine, and still risks operator fatigue and error. It is desirable to detect spike holes and automatically align the machine with the holes, after which the holes are filled using various injection materials.
In addition to detection of spike holes, systems are desired that are capable of detecting other railway features, including but not limited to spikes, anchors, and tie plates. Such systems may also allow for automated removal, replacement, and/or repositioning of these components.
In accordance with an aspect of the invention, a railroad tie maintenance vehicle for identifying and automatically sealing spike holes in a railroad tie includes a machine that moves along a railway having exposed spike holes in railroad ties. The machine includes an imaging device facing downwardly towards the railroad ties, and at least one injection device that is located adjacent to the imaging device and that is movable vertically and horizontally relative to the tie. A controller is coupled to the imaging device and the injection device and is operable, using information received from the imaging device, to identify the existence and locations of spike holes in the ties and to control the injection device to align an injector thereof with the identified spike holes and to dispense a chemical solution into identified spike holes.
The machine may include a spike hole detecting module, and an execution module. The detecting module includes instructions that, when executed, compare an image captured by the imaging device to a database of images of spike holes. The database may be prestored on the machine and/or in an external server, and may be either static or dynamic and updated with newly acquired images during a spike hole filling operation. The execution module includes instructions that, when executed, move the at least one injection device to be inserted into a spike hole and deposit the chemical solution into the spike hole.
In accordance with another aspect, the machine includes controls that drive the at least one injection device to move relative to the railway to position the at least one injection device in a longitudinal and lateral location proximate to the spike holes without operator input. For instance, a plurality of actuators may be provided that move the at least one injection device in longitudinal and lateral directions, as well as vertically, relative to the vehicle.
According to another aspect, the imaging device includes a camera, a hood that overlies the camera, as well as a light that is located adjacent to the camera and that illuminates the area in the field of view of the camera. The injection device is located directly adjacent to the hood, with a fixed lateral distance between the injection device and the imaging device. Thus, the hood is moved in the lateral direction once a spike hole is identified so that the injection device is located at the identified spike hole along the lateral axis. Further, a longitudinal actuator is configured to move the at least one injection device along the longitudinal axis so that the at least one injection device is located at the identified spike hole along the longitudinal axis.
The machine may have a mothership vehicle and a shuttle cart. The mothership vehicle includes a chassis and a plurality of wheels that support the chassis and that are configured to engage at least one rail. The shuttle cart is movably connected to the mothership vehicle by a physical or wireless tether. The shuttle cart also includes a chassis, as well as a propulsion assembly that is configured to propel the shuttle cart towards and away from the mothership vehicle. The injection device and imaging device are mounted on the shuttle cart.
The mothership vehicle may be configured to move at a substantially constant speed along a rail, whereas the shuttle cart travels at a varying speed along the rail relative to the mothership vehicle. More specifically, the shuttle cart frequently indexes to the next tie in the line, where it stops to fill spike holes, after which it moves along the rail to the next located railroad tie.
In accordance with another aspect, a method of operating a railroad tie maintenance machine is provided. The method includes moving the machine along a railway, capturing images of a plurality of railroad ties using an imaging device on the machine, using a controller on the machine to detect and identify at least one spike hole contained in a given railroad tie, under control of the controller, automatically moving an injection device on the machine to overlie the identified spike hole and dispensing a chemical solution into the spike hole. The moving of the machine may include moving a mothership vehicle along the railway and moving a shuttle cart relative to the mothership vehicle, and the shuttle cart supports the imaging device and the injecting device. The moving of the injection device may comprise actuating a plurality of actuators to move the injection device in lateral and longitudinal directions, and then vertically towards and then away from the spike holes. The controller may include or be in communication with one or more databases of images of spike holes, which may be static or may be dynamic and updated during spike hole filling operation. In addition to identifying spike holes, the controller may also be configured to detect other aspects of a railroad tie, such as imperfections in the tie, cracks in the tie, ballast rock or other foreign objects on the tie.
These and other aspects and objects of the present invention will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following description, while indicating preferred embodiments of the present invention, is given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the present invention without departing from the spirit thereof, and the invention includes all such modifications.
A clear conception of the advantages and features constituting the present invention, and of the construction and operation of typical mechanisms provided with the present invention, will become more readily apparent by referring to the exemplary, and therefore non-limiting, embodiments illustrated in the drawings accompanying and forming a part of this specification, wherein like reference numerals designate the same elements in the several views, and in which:
about line 11C-11C of
In describing the preferred embodiment of the invention which is illustrated in the drawings, specific terminology will be resorted to for the sake of clarity. However, it is not intended that the invention be limited to the specific terms so selected, and it is to be understood that each specific term includes all technical equivalents which operate in a similar manner to accomplish a similar purpose. For example, the words “connected”, “attached”, “supported”, or terms similar thereto are often used. They are not limited to direct connection but include connection through other elements where such connection is recognized as being equivalent by those skilled in the art.
The present invention and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments described in detail in the following description.
Referring to the general to the drawings, an automated tie filling machine 20 or vehicle for filling spike holes 22 in railroad ties 24 of a railway 25 is illustrated. Initially, the machine 20 will be described in connection with automated filling of spike holes 22 using a visual detection system that acquires still or video images of the spike holes 22. Additional examples will be provided below in which the visual detection system is used to detect other components associated with a railway.
The machine 20 typically will be used as part of a track re-laying operation in which a rail 26 is removed from the railway and replaced with a new rail. As part of this process, the spikes 27 holding the rail 26 to the tie 24 and holding the tie plate 28 to the rail 26 are removed by one or more machines (not shown) working ahead of the tie filling machine 20, leaving holes 22 in the ties 24, and one of the rails (the right rail in the illustrated example) is removed. Those holes 22 should be filled to preserve the integrity of the tie 24 and to provide solid surfaces into which new spikes (not shown) may be driven. Filling typically involves the injection of a chemical solution 30 (
Turning initially to
Still referring to
The shuttle cart 34 will now be further described with reference to
Additionally, the shuttle cart 34 includes first and second wheels 56 that rollably support the shuttle cart 34 on the rail 26 on the non-working side of the shuttle cart 34 opposite of the working side having the crawler 54. Further still, rail clamps 58 having first and second fingers or claws 60 may be positioned directly adjacent to each of the wheel guides 56. The rail clamps 58 are configured to be releasably adjusted between an opened position in which the first and second fingers 60 are spaced from the rail 26, and a closed position in which the first and second fingers 60 help secure the shuttle cart 34 to the rail 26.
In operation, the mothership vehicle 32 is typically propelled along the railroad at a substantially constant speed by its crawlers 38, while the shuttle cart 34 is able to move back and forth relative to the mothership vehicle 32 at different speeds by its crawler 54. The crawler 54 is powered by a hydraulic motor 55 using pressurized fluid supplied by the mothership vehicle 32, although other motors could similar be used to facilitate operation of the crawler 54. By allowing the shuttle cart 34 to be moveable relative to the mothership vehicle 32, the mothership vehicle 32 can move at a substantially constant speed, rather than starting and stopping every time a tie 24 is filled, which would result in frequent momentum changes to the mothership vehicle 32, and in turn the operator located therein. Of course, when the shuttle cart 34 is not filling spike holes 22, the mothership vehicle 32 and the shuttle cart 34 can move together along the railway at a substantially constant speed. When this occurs, the mothership vehicle 32 and the shuttle cart 34 can be mechanically coupled to one another.
Turning back to
It also should be noted that that a separate mothership vehicle 32 and shuttle cart 34 are not essential to the use of vision to identify, locate, and manipulate railway features. It is conceivable that the shuttle cart 34 could be eliminated in favor of one or more traditional workhead(s) on a vehicle that has the other characteristics of the mothership vehicle 32 disclosed herein. In this case, a portion or potentially all of the longitudinal movement of the components carried by the shuttle cart 34 of the illustrated embodiment could be mounted on a workhead that is moveably longitudinally of a frame on the vehicle. An example of such longitudinally movable workhead is disclosed in the '692 patent.
Returning again to the illustrated embodiment, the shuttle cart 34 includes one or more filler workheads 66 that are movable relative to the chassis 52, as best seen in
As best seen in
The injection wand assembly 68 is best shown in
Still referring to
Referring again to
In one embodiment, the CV software uses Tensorflow software, which is an open-source software library for machine learning and artificial intelligence that can be used across a range of tasks, with a focus on training and inference of deep neural networks. The Tensorflow software includes two stages: training and inference. In the training stage, a large amount of data is collected and stored. In the present invention, the data is images associated with specific portions of a railway, for instance, of ties and holes formed in the ties. This data is annotated and labeled using a labelling software to draw boxes around the ties and holes and label them respectively. Once the images have been labeled, the Tensorflow software then develops a model based on the provided data. In the present embodiment, an object detection model is generated, which is used to identify all instances of a particular object within a single image. The model is essentially an equation or process where an input type is entered (in the present invention, an image) and the software outputs the coordinates and confidence score of items detected in the image. The model is then shared across the machine 20. The CV software is configured to continuously refine the training process, such that the model is constantly updated to improve performance. The training stage may be started from scratch, or it can be built off of a pre-trained model. In one embodiment, a pre-trained Fastere-RNN model is used that begins the ability to perform object detection, which is further refined based on the objects being identified.
In the inference stage, the model that was previously created during the training stage is actually put to use. As will further be described below, an image is acquired from the imaging device 78 and is fed into the model. The model then identifies a specific type of item, such as a railroad tie 24 or a spike hole 22, and a confidence score is calculated. This information is transmitted to the controller 202, and various components of the machine 20 are appropriately moved and manipulated as will be described below.
Identification and filing of spike holes 22 will now be described.
In operation, the shuttle cart 34 is configured to move along the railway 25 and the mothership vehicle 32 as the mothership vehicle 32 is independently propelled along the railway 25 while images are acquired via the camera. The images may be still images and/or video frames. The acquired images are then analyzed by the CV software, against a model including a database of photographs of previously identified spike holes 22. As described, the model may be dynamic and be updated as the machine 20 moves along the railway 25, otherwise it may be static. Initially, the tie 24 is identified and images of the tie 24 are taken by the camera 78. The images are then transmitted to the controller 202 and/or to a remote server for analysis. In either event, the images are compared to the model including a database of images using the CV software described above in order to identify the existence and location of the ties 24 as well as the existence and location of the spike holes 22 in the ties 24. In many instances, a pattern of four spike holes 22 will be identified, although the CV software can similarly identify any number of spike holes 22 that may be present in a given tie 24.
While analyzing the images, the CV software is configured to specifically identify spike holes 22 as such, while also disqualifying other railway features, such as cracks in the tie, imperfections in the tie such as knots, stains, ballast rocks or other foreign objects located on a tie 24, and the like.
After the spike hole 22 locations have been confirmed, the cylinder 104 is driven to move the frame 101 along the rails 105 laterally relatively to the shuttle cart chassis 52 as shown by arrow 100, and motor 111 can be actuated to drive the belt 107 to move the injection wand assembly 68 longitudinally of the frame 101 as shown by arrow 102 until the injection wand assembly 68 directly overlies an identified spike hole 22. As can be seen in
The injection wand assembly 68 receives chemical solution from one or more storage tanks. In the illustrated embodiment, a solution 30 is a two-part resin solution. The two components are stored in large storage tanks 86, 88, on the mothership vehicle 32 and then transferred to the shuttle cart 34 through one or more of the hoses 64 of the tether 62. Of course, storage tanks of the solution could similarly be mounted on the shuttle cart 34. The hoses 64 may be housed within a hose protector 90 that extends laterally along the chassis 52 of the shuttle cart 34. The chemicals are then delivered to the mixing tube 72 through additional hoses 92 under the control of valves 94 that are automatically actuated by the controller 202. Each tank 86, 88 preferably has a sufficient capacity to permit continuous operation of the shuttle cart 34 for an extended period of time. A per-tank capacity of at least 50 gallons, and more preferably at least 100 gallons, is typical.
The solution 30 is typically deposited into the spike holes 22 for between approximately 600-2,000 milliseconds, and more typically approximately 800 milliseconds depending on a number of factors, including for instance the viscosity of the material, the temperature of the material and outdoor conditions, and the size of the spike hole 22. About 1.5 to 3 inches3, or between 25 and 50 milliliters, of solution, and more preferably typically approximately 2.15 inches3 or 35 milliliters of solution, are deposited within each spike hole 22 as a result of this process. Of course, an operator can use the interface to adjust the length of time that the filling solution is sprayed from the injection wand assembly 68 and the quantity of filling solution 30 that is deposited into the spike hole 22. In optimized conditions, the machine 20 preferably fills the spike holes associated with 15-20 ties per minute.
The machine 20 may also be equipped with additional components to ensure proper interaction between the mothership vehicle 32 and the shuttle cart 34. For instance, mothership vehicle 32 may include a sensor (not shown) that monitors the distance between the mothership vehicle 32 and the shuttle cart 34. Similarly, a rangefinder (not shown) may be used that measures the distance between the mothership 32 and the shuttle cart 34. Further still, the distance between the mothership vehicle 32 and shuttle cart 34 may also be monitored using a physical leash (not shown) that helps the operator monitor the distance between these components, and terminate movement if the measured distance surpasses a designated amount. Whether a sensor, rangefinder, or physical leash is used, machine operation can be controlled so that a maximum permissible distance, for instance between 5′ and 8′ and, more typically, between 6′ and 7′, between the mothership vehicle 32 and shuttle cart 34 cannot be exceeded. Additionally, the shuttle cart 34 may include safety bars 110, 112 on either end of the shuttle cart 34. Both safety bars 110, 112 are pivotable about the chassis 52 of the shuttle cart 34. When either bar 110, 112 is pivoted due to contact with another object, propulsion of the shuttle cart 34 can be terminated, and the operator can be notified to ensure appropriate safety measures are in place.
Turning to
Turning next to
Looking to
Next,
While the above description relates to filling spike holes 22 in a single tie 24, since there are two cameras 78, injection wand assemblies 68, and associated components, spike holes 22 in two ties 24 could be filled simultaneously by the first and second injection wand assemblies.
As described above, while the detailed description primarily discusses use of the machine 20 to identify the location of spike holes 22, the vision system described herein, or contemplated variations of it, could similarly be used to visually identify, locate, and act on other railway features. Such features include, but are not limited to, tie plates, spikes, rail anchors, and rail clips. The tool could act upon the feature by one or more of removing it, placing it, attaching it (such as by driving a spike or applying an anchor or clip), removing it (such as by pulling a spike or “spreading” anchors), etc.
For instance,
Although the best mode contemplated by the inventors of carrying out the present invention is disclosed above, practice of the present invention is not limited thereto. It will be manifest that various additions, modifications and rearrangements of the features of the present invention may be made without deviating from the spirit and scope of the underlying inventive concept.
Moreover, the individual components need not be formed in the disclosed shapes, or assembled in the disclosed configuration, but could be provided in virtually any shape, and assembled in virtually any configuration. Furthermore, all the disclosed features of each disclosed embodiment can be combined with, or substituted for, the disclosed features of every other disclosed embodiment except where such features are mutually exclusive.
It is intended that the appended claims cover all such additions, modifications and rearrangements. Expedient embodiments of the present invention are differentiated by the appended claims.
The present application claims priority on U.S. Provisional Patent Application Ser. No. 63/399,835, filed Aug. 22, 2022 and entitled Vision-Based Railway Component Characterization System and Method, the entirety of which is hereby incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 63399835 | Aug 2022 | US |
